The enhancement of catenary image with low visibility based on multi-feature fusion network in railway industry

Journal Pre-proof The enhancement of catenary image with low visibility based on multi-feature fusion network in railway industry Yuwen Chen, Bin Song, Xiaojiang Du, Nadra Guizani

PII: DOI: Reference:

S0140-3664(19)31273-3 https://doi.org/10.1016/j.comcom.2020.01.040 COMCOM 6161

To appear in:

Computer Communications

Received date : 27 September 2019 Revised date : 10 December 2019 Accepted date : 19 January 2020 Please cite this article as: Y. Chen, B. Song, X. Du et al., The enhancement of catenary image with low visibility based on multi-feature fusion network in railway industry, Computer Communications (2020), doi: https://doi.org/10.1016/j.comcom.2020.01.040. This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2020 Published by Elsevier B.V.

Journal Pre-proof

The enhancement of catenary image with low visibility based on multi-feature fusion network in railway industry Yuwen Chen1, Bin Song1*, Xiaojiang Du2, Nadra Guizani3 1 State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an 710071, China

of

2 Department of Computer and Information Sciences, Temple University, Philadelphia, PA 19122, USA 3 Gonzaga University, USA Corresponding author: Bin Song ([email protected]).

Jo

urn al P

re-

pro

Abstract In the Industrial Internet of Things (IIoT), the security and efficiency are indispensable. For the railway industry, the video from inspection vehicle would be influenced by various factors with low visibility and hard for high level vision task, such as the fault diagnosis of catenary system. In this paper, we propose a method based on the multi-feature fusion network to improve the quality and visual effect of the catenary images. The transmission map is learned from the multi-scale and multi-feature fusion network, which would learn coarse and fine details and combine the latent features. In the catenary image, the sky and non-sky regions are segmented through multiple accommodative thresholds to estimate the atmospheric light value. With the refinement of transmission map, the restored catenary image is obtained through the atmospheric scattering model. In the experimental results, it can be seen that the proposed method can improve the clarity of catenary image in haze. The quantitative evaluation shows that it has better visual effect compared with the other traditional methods. Key words: catenary system image, low visibility, deep learning 1. Introduction With the development of the Internet of Things (IoT), many objects in the environment could be connected through the network. The IoT realizes the real time and smart communication, identification, surveillance and so on [1-3]. The IoT technologies are applied in the industry to promote the manufacturing and industrial application. Since the Industrial Internet of Things (IIoT) is intelligent and integrated, which use machine learning such as deep learning to be reliable and highly performing. In the electric railway industry, the surveillance is important for the fault diagnosis and optimization of system. The pantograph catenary system supplies power to the vehicle, which plays an important role during the process of the train running. The check of the catenary system is critical and associated with the security and efficiency of the whole railway system. With the fast development of information processing and artificial intelligence technology, the image processing technology is widely used in the target detection, fault diagnosis, surveillance and so on, which is of great significance in the IIoT. In the surveillance and check of the catenary system, the quality of image or video from the inspection vehicle would influence the effect of high level vision task [4-6]. The catenary system which is built along the railway is usually in the field. The natural factors such as fog, haze, particle in the air and so on, would make the video and image from the inspection vehicle have low contrast and unclear details. These factors would make the security check of the catenary difficult [7-10]. In order to solve the problem of the poor visibility caused by haze in the video and image from the inspection vehicle, it is important to apply some proper image restoration techniques to process the surveillance video for the following detection of the catenary

Journal Pre-proof

Jo

urn al P

re-

pro

of

system. Since the preprocessing of image can provide assistance for the high level vision task, the technology of haze removal is applied widely and numerous dehazing algorithms are proposed. It can be divided into two aspects of these algorithms, which are image enhancement based algorithms and atmospheric scattering model based algorithms. The image enhancement based haze removal methods enhance the low quality image directly to improve the visual effect of image through mature image technologies, such as Retinex [11], histogram equalization [12], filtering[13], etc. Based on the Retinex theory, Zhou et al. [14] propose a single image dehazing method motivated by the Retinex theory, which adopts the image operated by the Gaussian filter to approximate the airlight and uses the optical model to obtain the haze free image. Patil et al. [15] apply a dark channel prior and fuzzy enhancement method to remove the haze in the image, which contains the fuzzy logic based histogram equalization technique. Park et al. [16] introduce an image refinement method through edge preserving smoothing, which is based on weighted least square. The local contrast is enhanced by multi-scale tone manipulation. Nair et al. [17] adopts a center surround filter to estimate the transmission map, which is applied in different color spaces and solve the haze image model. Du et al. [18] employ wavelet analysis to detect and remove the haze from the image. The image is decomposed into different spatial layers and the residual wavelet coefficients are used as a mask to recover the haze free image. The methods based on the image enhancement could improve the image visual effect through increasing the contrast of image, but the local region and color distortion phenomenon would appear. The atmospheric scattering model based methods mainly estimate and optimize the unknown parameters based on the image dehazing model. The haze free image is solved from the model. The representative algorithms are based on the priori knowledge, such as dark channel prior proposed by He et al. [19], color attenuation prior proposed by Zhu et al. [20] and so on [21-24]. According to the prior knowledge with haze imaging model, the transmission map would be refined and the haze free image could be restored. Shuai et al. [25] present a haze removal approach with wiener filtering, which transforms the image restoration into an optimization problem based on the dark channel prior. Chen et al. [26] adopt a gain intervention based filter to get the transmission map refinement and get the haze free image associated with the prior knowledge. However, the halo and block artifacts would be generated without transmission optimization in some sky or light color regions, and the color distortion phenomenon could exist in some prior knowledge based methods. With the development of machine learning technology, the learning based methods for the haze removal are applied widely. Tang et al. [27] present a learning framework to identify the best feature for image dehazing. The haze relevant features such as local max contrast, local max saturation, and hue disparity are used to build the image dehazing model. But the model would be influenced in some heavy fog and noise. The popular learning model convolutional neural network (CNN) is also applied in the image dehazing field. Cai et al. [28] propose a trainable end to end system to estimate the transmission. The deep convolutional neural network is established to generate the relevant features for removing haze from the image. Li et al. [29] propose a light weight convolutional neural network to generate the clean image directly, which can improve the performance of the high vision level task on hazy images. It can be seen that the various structures of convolutional neural networks could learn deep and latent features of image for the vision task. In this paper, the enhancement method for catenary image with low visibility is proposed. A multi-scale and multi-feature network based on the convolution kernel is designed. The coarse and

Journal Pre-proof

of

fine features are extracted and combined to learn the transmission map. The atmospheric optical value is estimated based on the multi-segmentation for the sky and non-sky regions through multiple accommodative thresholds. Based on the atmospheric scattering model, the restored catenary image is obtained with the refined transmission map and estimated atmospheric optical value. 2. Related work 2.1 The atmospheric scattering model The atmospheric scattering model [30] is widely used in the image dehazing methods. The model could express the haze degradation. The formulation can be denoted as follows, I x J x t x A 1 t x (1)

urn al P

re-

pro

t x 𝑒 (2) Where 𝐼 𝑥 is the haze image and 𝐽 𝑥 is the clear scene radiance. 𝐴 is the global atmospheric light, t(x) is the haze transmission matrix, d(x) is the depth of the scene and 𝛽 represents the scattering coefficient of the atmosphere. Numerous image haze removal algorithms adopt various ways to find the constrained relation or mapping relation between the haze image and transmission map or clear scene image. 2.2 CNN network CNN is one of the deep neural network, which develops rapidly in the image analysis, pattern recognition, and so on [31-33]. The architecture of CNN usually consists of input layer, convolutional layer, pooling layer, fully connected layer and output layer. The feature map is obtained by the convolutional layer which contains linear operation and activation function. The local interconnect and shared weight characteristics can reduce the quantity of the parameters in the network and learn the integral information from the local features. The multiple convolutional kernels can extract various features from the data. Combining with the pooling layer which has the ability of down sampling, the network can implement feature dimension reduction. The fully connected layer connects each neuron to all neurons of the previous layer and integrates the multiple feature maps. Considering the numerous parameters of the fully connected layer, the dropout layer or global average layer would be adopted alternatively.

Jo

3. The architecture of haze removal 3.1 The network based on multi-feature fusion The proposed network is shown in Fig.1, which is used to produce the estimation of the haze transmission map in the atmospheric scattering model. The size and number of convolutional kernels are illustrated in the figure. It consists of two parts, multi-scale feature extraction and multi-feature fusion. In order to keep the size of input and output consistent for fitting the haze degradation model and avoid the problem of information loss, the fully convolution operation is adopted without pooling and upsampling operation. For seeking the haze removal effect in different scale details, the multi-scale features are extracted. The dilated convolution [34] can capture multi-scale context through different dilation rates. The receptive field is expanded without extra parameters, which can provide more efficient ability of abstracting information. The structure of dilation convolution can be described in Fig.2. The layers conv3_2 and conv3_3 are dilated convolution in the network. The kernel is of size 3 and the dilation rate is 2 and 3 respectively.

of

Journal Pre-proof

pro

Fig.1. The structure of the network.

Fig.2. The illustration of dilated convolution.

The part of multi-feature fusion combines the multi-scale features to estimate the transmission map. The tensors are concatenated in some layers for fusing the various coarse and fine detail features. In

re-

concatenate1 layer, the original haze image is combined with other feature maps, which would preserve the information contained in the input image and ensure the refinement of features in the following layers. The other connections would compensate the loss of information. In addition, with the increasing depth of the network, the degradation problem would occur. The immediate concatenation could learn effectively in deep network. The concatenation layer can be presented as follows: 𝐻

urn al P

𝐹 ,𝐹 ,…𝐹

𝐻

max 0, 𝐻 ∗ 𝑊

(3)

𝑏

(4)

Where F is the various feature maps, k denotes the number of maps, H is the tensor in concatenation layer, m is the index of layer in the network. The concatenation layer is put into the next convolution layer 𝐻

, W is the convolution kernel, b is bias and max( .) denotes the activation function. The batch

normalization is applied after the convolutional layer. The activation function is ReLu [35]. The loss function is adopted as the mean square error function: 𝐿 θ

Where the 𝑡 𝑥

∑

‖𝑡 𝑥

𝑡∗ 𝑥 ‖

(5)

is the reconstructed transmission output of network, 𝑡 ∗ 𝑥

is the corresponding map

label, and n is the quantity of training set. The Adam [36] is applied as the optimizer for the training of network.

Jo

3.2 The recovery of clear image According to the image from the inspection vehicle, the catenary image usually contains two parts, the sky region and non-sky region. The catenary is always emerged in the sky region. In the refinement of unclear catenary image, it would be important to process the monitoring image with sky region. After getting the transmission map from the multi-feature fusion network, the refinement of transmission map and estimation of atmospheric optical value are necessary. For the visual effect of the clear image, the transmission map would have edge preserving features. In order to refine the transmission map further, the guided filter [37] is applied. For the guidance image, the gray input image operated by Gaussian kernel is used, which have smoothed details and retain the edges. The process is as follows:

Journal Pre-proof

𝐼

Where 𝐼

𝐺 𝐼

(6)

𝑎 𝐼 𝑏 , ∀𝑖 ∈ 𝑤 (7) 𝑡 ∑∈ 𝑏 𝑡 𝜀𝑎 (8) 𝐸 𝑎 ,𝑏 𝑎 𝐼 is the gray input, G is the Gaussian operation, the size of kernel is 5 and sigma is 0.5.

re-

pro

of

t is the output and 𝑡 is the input of filter, 𝑎 and 𝑏 are linear coefficients, 𝑤 is the local window, 𝜀 is adjustment coefficient, and 𝐸 is the cost function in the filter.

Fig.3. (a) The original haze image. (b) The histogram of the whole image. (c) The first segmentation result. (d) The result after fill operation (e) The histogram of the segmented sky region. (e) The second segmentation result. (f) The result of sky region judgement.

Jo

urn al P

For the estimation of atmospheric light value, the sky region and non-sky region are segmented with a simple method based on the multi-threshold. The original gray catenary image and the histogram of gray catenary image is shown in Fig.3 (a) and (b). The catenary image usually can be divided into sky region and non-sky region. It shows two wave peaks and one trough in the histogram. For the catenary images without sky regions, there are usually no distinct trough. The atmospheric evaluation based on the sky region would avoid the distraction from the bright object and noise in the foreground region. The sky region segmentation is operated based on multiple thresholds. The first threshold is calculated on the wave trough in the histogram of the whole image. For the threshold value at wave trough between two high wave peaks, the result is shown in Fig.3 (c). The general sky region is obtained. With the fill operation, the maximum connected region is considered as the approximate sky region which is integrated further in Fig.3 (d). But the catenary lines in the sky region is covered. In order to separate the catenary lines from the sky region, the second threshold is calculated. From the histogram of the segmented approximate sky region in Fig.3(e), the catenary lines contain less pixels and lower intensity than the sky pixels. The sky region is usually large and bright. The pixel value which appears firstly with distinct increased quantity compared with that of its smaller pixel value would be chosen as the second threshold. The second segmentation of the sky region is shown in Fig.3(f). It can be seen that the catenary regions are segmented and some sky regions are dropped by the second segmentation. As the sky region is always homogeneous without edge saliency information, the gradient and saliency information is used to judge the sky region further. It can be calculated as follows: Φ 𝐼 𝑥 (9) 𝑃 𝑥 𝑃 𝑥

Φ

Ψ 𝑃 𝑥 ,𝑥 ∈ r

(10)

Journal Pre-proof

𝑆 𝑥

|𝑆

𝐼

𝑥

𝑆

𝐼

𝑥 |

(11)

Θ 𝑃 𝑥 ,𝑃 𝑥 ,𝑥 ∈ r (12) 𝑅 𝑥 𝑖𝑓 𝑆 𝑥 𝜖, 𝑥 ∈ 𝑅 (13) (14) 𝑃 Υ 𝑃 ,𝑅 Where 𝛷 is the segment operation. The first segmentation result based on the whole image is 𝑃 , and the second segmentation result based on the sky region 𝑟 is 𝑃 . 𝛹 is the fill function. 𝑆 is the convolution function with Gaussian kernel G. Here the G1 is the kernel with size 5 and sigma 0.5, and the G2 is an average kernel, which could be used to calculate the saliency information S. R is the different region in the segmentation of sky region. The region which have smaller saliency value than 𝜖 would be judged by the sky region of R. 𝜖 is decided by the sum result of mean and half of standard deviation of saliency information in region R. The final segmented sky region is P is the second segmented region 𝑃 with the region 𝑅 . The final segmentation is shown in Fig.3 (g). The atmospheric light value evaluation is calculated as follows: (15) Δ 𝛵 𝑓 𝐼 𝑀 𝐼 (16) 𝐴 ∈

pro

of

𝑅 𝑥 𝑅 𝑥

𝐴

re-

Where 𝑓 is a mean filter to calculate the mean intensity value in local region with size 5. P means the sky region. T means the selected region which has the one percent bright pixels in the sky region. c is the channel of image I, 𝑀 is the operation of selecting the medium value of the pixels in the region 𝛥. For the catenary image without sky region, the atmospheric light value is calculated as follows: 𝑀 𝑓 𝑆

𝐼

(17)

urn al P

Where c is the channel of image I. 𝑆 is the same function in the above Eq. (11). 𝑓 is the minimum filter with the window size 5. 𝑀 is the average operation with the one percent of the brightest pixel values. After obtaining the transmission map 𝑡 𝑥 and atmospheric light value A, according to the Eq.(1), the restored image would be obtained as follows: 𝐽 𝑥

,

𝐴

(18)

Where 𝑡 is set to avoid the situation that 𝐽 𝑥 𝑡 𝑥 is close to zero when too small transmission value. Here 𝑡 is set as 0.1[19]. The process of the recovery of clear image is presented in following block. The process of the recovery of clear image

Jo

Input: The transmission map obtained from the network in Section 3.1 Output: The restored image 1. The refinement of the transmission map is calculated by Eq. (6-8). 2. The estimation of atmospheric light value: a. The first and second segmentation operation is computed by Eq. (9-10). b. The final sky region segmented is computed by Eq. (11-14) c. The estimated value of atmospheric light is computed by Eq. (15-17). 3. The restored image is calculated by Eq. (18). With the transmission map learned from the multi-feature fusion network, the refinement of the transmission map could improve the details of the map further. According to the characteristics

Journal Pre-proof

Jo

urn al P

re-

pro

of

of the catenary image, the atmospheric light value estimation is calculated based on the multiple segmentation of the sky region. The restored catenary image would have the improvement of the clarity. 4. Experiments The simulated experiments are implemented on the synthetic and natural images. The baselines methods are introduced for comparison and assessment. To train the network, there are about 20000 triples of haze image, clear image and transmission map which are synthesized based on the NYU depth[38], Make3D[39-41] and Middlebury Stereo Dataset[42-44]. About 2000 clear catenary images selected from the inspection videos are also used in the training stage. The depth maps are computed through depth estimation model [45]. The synthesized haze images are computed by equation (1) and (2). The synthesized parameters of A and 𝛽 in the equations are in the range of 0.7 to 1 and 0 to 1.5. The batch size is set to 16. The images are selected randomly as 240 320 image patches to put into the network. The model is implemented on Pytorch 0.4. The initial learning rate is 0.001 in the first 20 epochs and divided by 10 after every 10 epochs. The epoch is set as 40 and the training time is about 10h. The experimental platform is equipped with Intel Core i7 CPU and NVIDIA GeForce GTX GPU. The images used in the comparisons are selected randomly in the foggy catenary images from the inspection videos and synthetic haze catenary images respectively. The baseline methods are dark channel prior method [19], improved visibility restoration method [24], dehaze net method[28], AOD net method[29], which are denoted as M1, M2, M3, M4. The proposed method is denoted as M5. The original catenary images are shown in Fig.4. When the weather is cloudy or fog, the visibility would be low and the scene in the image is blurry. These images would disturb the high level vision task such as fault diagnosis of the catenary system. The original low visible images are in the first column. Fig.4(b)-(f) are the results of M1-M5. The each row is the results corresponding to the original image. The quantitative evaluation is based on subjective and objective assessments. The subjective evaluation given by the visual observation of human visual system. For the objective evaluation, there is no reference image for the catenary image to evaluate the restored result. So the objective measurement is adopted for objective evaluation metrics, which are information entropy, average gradient, rate of new visibility and mean of visual edge gradient [46]. The information entropy can reflect the average information amount of the image. The large value means the rich information. The average gradient can reflect the detail and edge features. The large value is associated with the plentiful texture. The rate of new visible and mean of visual edge gradient can express the changes of visible edge quantity. These metrics would present the processed image quality in some degree. In Fig.4, it can be seen that the methods M1 and M2 with prior knowledge are not suitable for the haze catenary images. The catenary images usually contain large sky area. The scenes in the images are various and complicated. The results of M1 in Fig.4 (b), the sky regions have heterogeneous gray or dark color, which present unnatural blocks. The foregrounds in the third and sixth images are dark and dim, which make the image hard to be recognized. In Fig.4(c), the results of M2 exhibit severe distortion. The hues of result images tend to be dark. Many unnatural blocks are introduced in the image. At the edge region of image, especially the margin between the sky and foreground, the white areas exist. The methods M3, M4, and M5 with learning process present better visual effects. The images have less color distortion and unnatural area. In Fig.4(d), it can be seen that the sky regions are restored without dark regions. But in some low light regions, such as

Journal Pre-proof

Jo

urn al P

re-

pro

of

the mountain area in the last result image, it presents dark without rich color. The results of M4 in Fig.4(e), the foregrounds in the images are bright, but the color of tree is distorted to some extent. The sky regions trend to be cool and blue color. The colors of images seem to be unbalanced. The results of the proposed method M5 have better visual effect than the other results in Fig.4(f). The colors of the sky region and the foreground region are balanced with without dark blocks. The whole images are bright and have clarity improvement. The details of the images are presented clearly. The results with suitable visual effect and less color distortion are adopted to compute the objective evaluation metrics in the Table 1, and 2. The results of M1 and M2 are not involved, which contain distinct color distortion. Some of the metrics are computed by the edge information, the distortion would influence the value of metrics. For example, the unnatural edges introduced by the distinct distortions in the results of M1 and M2 could magnify the values of metrics. Here, the objective evaluation is computed in the results with small distortion degree. The information entropy, average gradient, rate of new visibility and mean of visual edge gradient, are denoted as E, AG, e and r respectively. The comparisons are based on the methods which have better visual effects. The metrics based on the result images are presented in the tables. It can be seen that the proposed method can have more abundant information and improve the visible details with less color distortion.

Fig.4. The column (a) are the original catenary image with low visibility. The column (b)-(f) are the results of

methods M1-M5.

Journal Pre-proof

Table 1. The values of AG and E of each method. M4

M5

(1)

3.6325

3.8901

(2)

2.2691

(3)

M3

M4

M5

4.2478

6.8631

6.4465

6.9151

3.1113

3.2980

6.2848

6.3945

6.4878

3.0410

3.4547

3.6412

6.8199

6.5630

6.8216

(4)

2.5721

3.5633

3.4447

6.7597

6.9494

6.9602

(5)

3.9234

4.0746

4.9448

6.9886

6.6744

7.0069

(6)

4.0516

4.1948

4.3289

7.0336

6.9349

7.1894

Table 2. The values of e and r of each method. M3

M4

M5

(1)

0.5908

0.6988

0.5940

(2)

2.2371

4.7079

4.8505

(3)

1.0586

1.2449

1.2829

(4)

3.1548

5.6181

5.8148

(5)

0.3523

0.5309

(6)

1.4851

1.0193

r

M3

M4

M5

1.6017

1.8001

1.8105

2.1257

3.2097

3.7983

1.5646

1.9099

1.7832

2.1372

3.1402

3.6695

0.5928

1.6440

1.8065

1.9170

1.4867

1.5707

1.7484

1.7538

re-

e

E

of

M3

pro

AG

urn al P

According to the simulated experiments, it can be seen that the proposed method has good effect on the enhancement of catenary image. The details in the image are presented clearly with removing the blur information. The color of the image is plentiful and balanced, which makes the catenary image have good visual effect. 5. Conclusion In this paper, a multi-scale and multi-feature fusion network is presented for the clarity improvement based on the catenary image with low visibility, which would improve the quality of image for enhancing and optimizing the vision task in the railway industrial internet of things. The coarse and fine features are abstracted and fused to learn the transmission map. The transmission map is refined further to preserve the edges. The multiple accommodative thresholds are adopted to segment the sky and non-sky regions for evaluating the atmospheric light value. The restored catenary image is obtained through the atmospheric scattering model. Experimental results show that the proposed method can improve the clarity and visual effect of the catenary image. The objective evaluation compared with other traditional algorithms present the effectiveness of the proposed method.

Jo

Acknowledgements This work has been supported by the National Natural Science Foundation of China (No.61772387), the Fundamental Research Funds of Ministry of Education and China Mobile (MCM20170202), the National Natural Science Foundation of Shaanxi Province (Grant No.2019ZDLGY03-03) and also supported by the ISN State Key Laboratory.

Journal Pre-proof

Jo

urn al P

re-

pro

of

References [1] H. Boyes, B. Hallaq, J. Cunningham, T. Watson, The industrial internet of things (IIoT): An analysis framework, Computers in industry, 101(2018) 1-12. [2] Z. Zhou, J. Feng, B. Gu, B. Ai, S. Mumtaz, J. Rodriguez, M. Guizani, When Mobile Crowd Sensing Meets UAV: Energy-Efficient Task Assignment and Route Planning, IEEE Transactions on Communications, 66(11)(2018) 5526-5538. [3] Z. Zhou, M. Dong, K. Ota, G. Wang, L. T. Yang, Energy-Efficient Resource Allocation for D2D Communications Underlaying Cloud-RAN-Based LTE-A Networks, IEEE Internet of Things Journal., 3(3)(2016) 428-438. [4] H. Hu, B. Tang, X. Gong, W. Wei, H. Wang, Intelligent Fault Diagnosis of the High-Speed Train With Big Data Based on Deep Neural Networks, IEEE Transactions on Industrial Informatics 13(4)(2017) 2106-2116. [5] Z. Liu, L. Wang, C. Li, Z. Han, A High-Precision Loose Strands Diagnosis Approach for Isoelectric Line in High-Speed Railway, IEEE Transactions on industrial informatics 14(3)(2018) 1067-1077. [6]J. Yin, W. Zhao, Fault diagnosis network design for vehicle on-board equipments of high-speed railway: A deep learning approach, Engineering Applications of Artificial Intelligence 56(2016) 250-259. [7]J. Chen, Z. Liu, H. Wang, A. Núñez, Z. Han, Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network, IEEE Transactions on Instrumentation and Measurement 67(2) 2018 257-269. [8] C .Wu, Z. Liu, H. Jiang, Catenary image enhancement using wavelet-based contourlet transform with cycle translation, Optik - International Journal for Light and Electron Optics 125(15)( 2014) 3922-3925. [9] I. Aydin, M. Karakose, E. Akin, A New Contactless Fault Diagnosis Approach for PantographCatenary System Using Pattern Recognition and Image Processing Methods, Advances in Electrical & Computer Engineering 14(3)( 2014) 79-88. [10] X. Ji, J. Cheng, J. Bai, T. Zhang, M. Wang, Real-time enhancement of the image clarity for traffic video monitoring systems in haze, In: Proceedings of the 7th International Congress on Image and Signal Processing, 2014. [11] T. Yu, X. Meng, M. Zhu, M. Han, An Improved Multi-scale Retinex Fog and Haze Image Enhancement Method, In: Proceedings of the International Conference on Information System and Artificial Intelligence, 2016. [12] H. Xu, G. Zhai, X. Wu, X. Yang, Generalized equalization model for image enhancement, IEEE Transactions on Multimedia 16(1)( 2014) 68-82. [13] Z. Li, J. Zheng, Single Image De-Hazing Using Globally Guided Image Filtering, IEEE Transactions on Image Processing 27(1)( 2018) 442-450. [14] J. Zhou, F. Zhou, Single image dehazing motivated by Retinex theory, In: Proceedings of the 2nd International Symposium on Instrumentation and Measurement, Sensor Network and Automation, 2013. [15] V. S. Patil, R. H. Havaldar, Haze removal and fuzzy based enhancement of image, In: Proceedings of the IEEE International Conference on Computational Intelligence and Computing Research, 2016. [16] D. Park, D. K. Han, H. Ko, Single image haze removal with WLS-based edge-preserving

Journal Pre-proof

Jo

urn al P

re-

pro

of

smoothing filter, In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2013. [17] D. Nair, P. Sankaran, Color image dehazing using surround filter and dark channel prior, Journal of Visual Communication and Image Representation 50(2018) 9-15 [18] Y. Du, B. Guindon, J. Cihlar, Haze detection and removal in high resolution satellite image with wavelet analysis, IEEE Transactions on Geoscience and Remote Sensing 40(1)( 2002) 210217. [19] K. He, J. Sun, X. Tang, Single Image Haze Removal Using Dark Channel Prior, IEEE Transactions on Pattern Analysis and Machine Intelligence 33(12)( 2011) 2341-2353. [20] Q. Zhu, J. Mai, L. Shao, A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior, IEEE Transactions on Image Processing 24(11)( 2015)3522-3533. [21] C. H. Yeh, L. W. Kang, M. S. Lee, C. Y. Lin, Haze effect removal from image via haze density estimation in optical model, Optics Express 21(22)( 2013)27127-41. [22] J. Zhang, Y. Cao, S. Fang, Y. Kang, C. Chen, Fast Haze Removal for Nighttime Image Using Maximum Reflectance Prior, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. [23] L Li, H Sang, G Zhou, N Zhao, D Wu, Instant haze removal from a single image, Infrared Physics & Technology 83(2017) 156-163. [24] J.-P. Tarel, N. Hautière, A. Cord, D. Gruyer, H. Halmaoui, Improved visibility of road scene images under heterogeneous fog, In: Proceedings of IEEE Intelligent Vehicle Symposium, San Diego, USA, 2010. [25] Y. Shuai, R. Liu, W. He, Image Haze Removal of Wiener Filtering Based on Dark Channel Prior, In: Proceedings of the 8th International Conference on Computational Intelligence & Security IEEE, 2013. [26] B. Chen, S. Huang, F. Cheng, A High-Efficiency and High-Speed Gain Intervention Refinement Filter for Haze Removal, Journal of Display Technology 12(7)(2016)753-759. [27] K. Tang, J. Yang, J. Wang, Investigating Haze-Relevant Features in a Learning Framework for Image Dehazing, In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition IEEE, 2014. [28] B. Cai, X. Xu, K. Jia, C. Qing, D. Tao, DehazeNet: An End-to-End System for Single Image Haze Removal, IEEE Transactions on Image Processing 25(11)( 2016) 5187-5198. [29] B. Li, X. Peng, Z. Wang, J. Xu, D. Feng, AOD-Net: All-in-One Dehazing Network, In: Proceedings of the IEEE International Conference on Computer Vision, 2017. [30] S. G. Narasimhan, S. K. Nayar, Vision and the atmosphere. International Journal of Computer Vision 48(3)( 2002)233-254. [31] F. Lateef, Y. Ruichek, Survey on semantic segmentation using deep learning techniques. Neurocomputing 338(2019)321-348. [32] B. Jan, H. Farman, M. Khan, M. Imran, I. U. Islam, A. Ahmad, S. Ali, G. Jeon, Deep learning in big data Analytics: A comparative study. Computer & Electrical Engineering 75(2019)275-287. [33]J. Guo, B. Song, F.R. Yu, Y. Chi, C. Yuen, Fast Video Frame Correlation Analysis for Vehicular Networks by Using CVS- CNN. IEEE Transactions on Vehicular Technology 68(7)(2019)62866292. [34] F. Yu, V. Koltun, Multi-scale context aggregation by dilated convolutions, arXiv:1511.07122, 2015.

Journal Pre-proof

Jo

urn al P

re-

pro

of

[35] V. Nair, G. E. Hinton, Rectified linear units improve restricted Boltzmann machines, In: Proceedings of the 27th International Conference on International Conference on Machine Learning, 2010, 807–814. [36] D. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv:1412.6980, 2014. [37] K. He, J. Sun, X. Tang, Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(6)( 2013)1397-1409. [38] N. Silberman, D. Hoiem , P. Kohli, R. Fergus, Indoor Segmentation and Support Inference from RGBD Images, In: Proceedings of the 12th European conference on Computer Vision, (2012)746760. [39] A. Saxena, M. Sun, A. Y. Ng, Make3D: learning 3D scene structure from a single still image, IEEE Transactions on Pattern Analysis and Machine Intelligence 31(5)( 2009)824-840. [40] A. Saxena, S. H. Chung, A. Y. Ng, Learning Depth from Single Monocular Images, In Proc. NIPS 2005. [41] A. Saxena, S. H. Chung, A. Y. Ng, 3-D Depth Reconstruction from a Single Still Image, In Proc. International Journal of Computer Vision, (2007). [42] D. Scharstein, R. Szeliski, A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, International Journal of Computer Vision 47(1)(2002)7–42. [43] H. Hirschmuller, D. Scharstein. Evaluation of cost functions for stereo matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2007) 1-8. [44] D. Scharstein, R. Szeliski, High-accuracy stereo depth maps using structured light. In: Proceedings of the Computer Vision and Pattern Recognition, 1(2003), 195-202. [45] F. Liu, C. Shen, G. Lin, I. Reid, Learning depth from single monocular images using deep convolutional neural fields. IEEE transactions on pattern analysis and machine intelligence. 38(10) (2016) 2024-2039. [46] N. Hautière, J.-P. Tarel, D. Aubert, É. Dumont. Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Analysis & Stereology 27(2) (2008)87-95.

Journal Pre-proof We confirm that the manuscript has been approved by all named authors. There is no conflict of interest of any authors in relation to the submission. This paper has not been submitted elsewhere for consideration of publication.

Jo

urn al P

re-

pro

of

All authors as follows: Bin Song (corresponding author) Xiaojiang Du Nadra Guizani Yuwen Chen

Journal Pre-proof Conflict of interest No conflict of interest.

of

The authors, Bin Song (corresponding author)

pro

Xiaojiang Du Nadra Guizani

Jo

urn al P

re-

Yuwen Chen