Adaptive total variation L1 regularization for salt and pepper image denoising

Journal Pre-proof Adaptive Total Variation L1 Regularization for Salt and Pepper Image Denoising Dang Ngoc Hoang Thanh, Le Thi Thanh, Nguyen Ngoc Hien, Surya Prasath

PII:

S0030-4026(19)31575-X

DOI:

https://doi.org/10.1016/j.ijleo.2019.163677

Reference:

IJLEO 163677

To appear in:

Optik

Received Date:

30 July 2019

Revised Date:

8 October 2019

Accepted Date:

26 October 2019

Please cite this article as: Thanh DNH, Thanh LT, Hien NN, Prasath S, Adaptive Total Variation L1 Regularization for Salt and Pepper Image Denoising, Optik (2019), doi: https://doi.org/10.1016/j.ijleo.2019.163677

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier.

Adaptive Total Variation L1 Regularization for Salt and Pepper Image Denoising

Dang Ngoc Hoang Thanh1*, Le Thi Thanh2, Nguyen Ngoc Hien3, Surya Prasath4, 5, 6, 7 1

Department of Information Technology, Hue College of Industry, Hue 530000 VN

2

Department of Basic Sciences, Ho Chi Minh City University of Transport, Ho Chi Minh 700000 VN

3

Center of Occupational Skills Development, Dong Thap University, Dong Thap 870000 VN Division of Biomedical Informatics, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH 45229 USA

4 5

Department of Pediatrics, University of Cincinnati, OH USA

6

of

Department of Biomedical Informatics, College of Medicine, University of Cincinnati, OH 45267 USA

7

ro

Department of Electrical Engineering and Computer Science, University of Cincinnati, OH 45221 USA

re

-p

Abstract: In this paper, we propose an adaptive total variation (TV) regularization model for salt and pepper denoising in digital images. The adaptive TV denoising method is developed based on the general regularized image restoration model with L1 fidelity for handling salt and pepper noise model. An estimation for regularization parameter is also proposed based on the characteristics of the salt and pepper noise. We implement the proposed adaptive TV-L1 regularization model efficiently for image denoising using the primal dual gradient method. In the experiments, the full-reference image quality assessment metrics are used for evaluating denoising quality across various noise levels in different synthetic and real images. The denoising results are compared to other similar salt and pepper denoising methods and our results indicate we obtain artifact free edge preserving restorations.

1.

lP

Keywords: Image Denoising, Salt and Pepper Noise, Total Variation, Image Restoration, Primal Dual Gradient, Adaptive Image Denoising, Image Quality Assessment.

Introduction

Jo

ur na

Image restoration [1, 2, 3, 4, 5, 6] is a common problem in image processing. Real images are usually corrupted by many factors such as noise, dust, scratches etc. The distortion reduces effectiveness of other image processing tasks such as image segmentation [7] and pattern recognition [8]. Therefore, image restoration plays a vital role to improve image quality. Image denoising is one of the most important image restoration problems. Noise usually appears in digital images taken by various imaging systems such as digital cameras, CT/X-ray/MRI scanners, microscopies, telescopes etc. There are some popular types of noises: the Gaussian noise [9], the Poisson noise [10], the impulse noise [11, 12], and the Gaussian-Poisson noise [13]. The salt and pepper (SnP) noise is a simple type of the impulse noise [14]. The SnP noise can be caused by sharp and sudden disturbances in the image signal [14]. The SnP noise has only two gray values: white pixel (the maximum gray value) and black pixel (the minimum gray value). The white pixel is also known as a salt pixel and the black pixel – a pepper pixel. To remove the SnP noise, there are many approaches used. In this paper, we only focus on approaches that are not based on machine learning. They include regularization, nonlinear filters, wavelet analysis, mathematical transformations and principal component analysis (PCA). For nonlinear filters, median filters are an effective approach to treat SnP noise. The classical Median Filter (MF) is presented in the work of Lim [15]. However, MF is only effective for low-density SnP noise. For medium-density and high-density noises, MF cannot remove noise completely and usually creates many defects. To remove the drawback, Adaptive Median Filter (AMF) [16] and Adaptive Center-Weighted Median Filter (ACWMF) [17] were proposed. Although AMF and ACWMF work more effectively than MF, they still cause many defects for the cases of medium-density and high-density noises. In recent years, the Based Pixel Density Filter (BPDF) was proposed by Erkan et al [12] is an effective filter for SnP noise. However, for the cases of high-density noise, BPDF usually creates raindrop effects that destroy image structure. Otherwise, BPDF does not work effectively on image regions containing gray values near the gray values of noise (i.e., the gray values are usually from 0 to 10 or 250 to 255 for 8-bit grayscale images). Wiener Filter [15] is another well-known nonlinear filter to treat SnP noise. However, Wiener Filter only can remove low-density SnP noise. For medium-density and high-density noise, Wiener Filter works ineffectively, and usually noise still remains. Therefore, there are some adaptive Wiener Filters was proposed to improve noise reduction performance [18, 19]. The bilateral filter [20] is a nonlinear filter designed primarily for removing the Gaussian noise. To remove the impulse noise as well as SnP noise, an adaptive bilateral filter (ABF) was proposed [21]. ABF works well on low-density and medium-density SnP noise.

Adaptive Total Variation L1 Regularization for Salt and Pepper Denoising

2.1. Image Restoration Problem and Total Variation

re

2.

-p

ro

of

Another class of approaches for SnP denoising is based on wavelet analysis. Noise can be removed by various methods such as Empirical Bayes [22], False Discovery Rate [23] (FDR), Stein's Unbiased Risk Estimate [24], Minimax Estimation [25] or Universal Threshold [25]. The methods can be implemented on various wavelets domain such as Haar, Coiflet, Symlet etc. Regularization is a novel approach acquiring many achievements in recent years. Regularization is especially effective to remove the Gaussian noise, the Poisson noise and the Gaussian-Poisson noise. There are some works applying regularization approach for the impulse noise, including SnP noise. Hemant et al. proposed Generalized Synthesis and Analysis Prior Algorithms [11] (GSAP). Aggarwal et al [26] proposed an impulse noise removal model based on the spatio-spectral TV regularization (SSTV). SSTV can be used to remove the Gaussian noise or the impulse noise alone as well as the combined Gaussian-impulse noise. Chan and Esedoglu [27] proposed a SnP denoising method based on Total Variation with L1 norm (TVL1). Although the TVL1 method is effective for the SnP noise, the corresponding regularization parameter estimation is still an open problem. In the article, our contributions focus on: reformulating the denoising model for the SnP noise based on the first-order Total Variation; considering the general image restoration problem to formulate the explicit SnP denoising model; proposing an estimation method for regularization parameter based characteristics of the SnP noise; and proposing an algorithm to solve the explicit model based on the primal dual gradient method [28, 29]. Our approach is a new model for adaptive Total Variation with L1 norm where our adaptive regularization parameter is computed based on the mean value of corrupted pixels thereby guiding the edge preserving total variation effectively to reduce the SnP noise. In the experiments, we test the proposed denoising method on synthetic images as well as on real natural images of the UC-Berkeley dataset [30]. Denoising quality is evaluated based on the full reference image quality assessment metrics: The Peak signal-to-noise ratio (PSNR) and the Structural similarity (SSIM) metrics. We compare the denoising results of the proposed model with existing methods representing different domains: median filters (MF, ACWMF, BPDF), Wiener Filter (Wiener), Adaptive Bilateral Filter (ABF), Wavelet Denoising Method with FDR and Coiflet (Wavelet), regularization-based models (GSAP, TVL1 and SSTV). Experimental results on synthetic as well as real images indicate that we obtain high-quality results visually and quantitatively across various SnP noise levels. The rest of this paper is organized as follows. Section 2 presents our adaptive TV-L1 based SnP denoising model, along with a regularization parameter estimation. Section 3 presents experimental results and comparisons to related denoising methods visually and quantitatively. Finally, Section 4 concludes the paper.



Jo

ur na

lP

Let 𝑢(𝐱), 𝑣(𝐱) ∈ ℝ – be a grayscale restored image and a grayscale corrupted (noisy) image, respectively, where 𝐱 = (𝑥1 , … , 𝑥ℎ ) ∈ Ω ⊂ ℝℎ is a pixel location, Ω is an image domain. The image restoration problem by Total Variation [31, 13] for a grayscale image has the following form: 𝜆 𝑝 (1) 𝑢 = argmin (𝐸(𝑢) + ∫ |𝐊𝑢 − 𝑣|𝑝 𝑑𝐱), 𝑝 Ω 𝑢 where |∙|𝑝 is 𝐿𝑝 -norm, 𝐸(∙) is an energy functional, 𝐊 is a filtering operator, 𝜆 is a regularization parameter. We note that a color image can be considered as a combination of various grayscale channels. Hence, this model holds true with color images. However, to simplify notations and evaluations with other related SnP denoising models, we only consider the case of 2D image here, i.e., ℎ = 2, 𝐱 = (𝑥1 , 𝑥2 ). For color images, the models can be written out in similar fashion. In the case of 𝑝 = 2, the model (1) is used for effective restoration of an image corrupted by the Gaussian noise. If 𝑝 = 1, the model (2) is effective for SnP denoising. The filtering operator 𝐊 has various forms depending on the types of restoration. For the image deconvolution problem, 𝐊 can be a convolution kernel such as the Gaussian kernel. For the denoising problem considered in the article, operator 𝐊 is identity operator 𝐈, i.e., 𝐊 ≡ 𝐈. For the term of the energy functional 𝐸(𝑢), we have some following important kinds: First-order Total Variation [32]: 𝐸(𝑢) = ∫ |∇𝑢|𝑑𝐱

(2)

Ω



Nonlocal Total Variation [33]: 2

𝐸(𝑢) = ∫ √∫ (𝑢(𝐱) − 𝑢(𝐲)) 𝜔(𝐱, 𝐲)𝑑𝐲 𝑑𝐱 , Ω

(3)

Ω

where 𝐲 = (𝑦1 , 𝑦2 , … ) ∈ Ω; 𝜔(𝐱, 𝐲) – the nonlocal weight to measure similarity of patches centered at the pixels 𝐱 and 𝐲.  Higher-order Total Variation [34]: 𝐸(𝑢) = ∫ |∇𝛾 𝑢|𝑑𝐱 , ∇𝛾 𝑢 = ∇(∇𝛾−1 𝑢), 𝛾 = 2,3, … Ω

(4)



Fractional-order total variation [35]: 𝐸(𝑢) = ∫ |∇𝛾 𝑢|𝑑𝐱 , 𝛾 ∈ (0,1)

(5)

Ω



Mumford-Shah [5]: 𝐸(𝑢) = ∫ |∇𝑢|2 𝑑𝐱 + 𝜅𝐻1 (Γ),

(6)

Ω\Γ

where Γ – set of edge pixels, 𝐻1 – the 1D Hausdorff measure, parameter 𝜅 > 0. 

Euler-Elastica [36]: 𝐸(𝑢) = ∫ (𝑎 + 𝑏 (∇ Ω

∇𝑢 2 ) ) |∇𝑢| 𝑑𝐱 , |∇𝑢|

(7)

of

where 𝑎, 𝑏 > 0. Denoising by the first-order Total Variation (2) can cause artifacts. Other kinds (3)-(7) can avoid this defect. However, the artifacts usually appear during restoring image from complex-structured noise such as the Gaussian noise or the Poison noise.

ro

2.2. Adaptive TV-L1 Regularization for Salt and Pepper Denoising SnP noise on an image has the following form [2]:

0 𝑝 = { |𝑢 − 𝛿𝑚𝑎𝑥 |𝑝

if 𝐱 is not noisy if 𝐱 is a salt pixel

(8)

|𝑢 − 𝛿𝑚𝑖𝑛 |𝑝𝑝

if 𝐱 is a pepper pixel.

0 |𝑢 − 𝑣|1 = {𝛿𝑚𝑎𝑥 − 𝑢 𝑢 − 𝛿𝑚𝑖𝑛

if 𝐱 is not noisy if 𝐱 is a salt pixel if 𝐱 is a pepper pixel.

(9)

0 |𝑢 − 𝑣|𝑝𝑝 = { (𝛿𝑚𝑎𝑥 − 𝑢)𝑝 (𝑢 − 𝛿𝑚𝑖𝑛 )𝑝

if 𝐱 is not noisy if 𝐱 is a salt pixel if 𝐱 is a pepper pixel.

(10)

If 𝑝 ≥ 2, the expression (8) is equivalent to the follows:

Jo



|𝑢 −

𝑝 𝑣|𝑝

If 𝑝 = 1, we can rewrite (8) easily as follows:

ur na



lP

re

-p

𝛿 with probability 𝑞1 𝜂(𝐱) = { 𝑚𝑎𝑥 . 𝛿𝑚𝑖𝑛 with probability 𝑞2 The value 𝛿𝑚𝑎𝑥 is a gray value of salt pixels (white pixels), and the value 𝛿𝑚𝑖𝑛 is a gray value of pepper pixels (black pixels). For an 8-bit grayscale image, 𝛿𝑚𝑖𝑛 = 0, 𝛿𝑚𝑎𝑥 = 255. The sum 𝑞 = 𝑞1 + 𝑞2 is called to be a noise level. As we abovementioned, selection of the energy function 𝐸(𝑢) can avoid artifacts. However, for the SnP noise, artifacts seldom appear in the denoised result. Therefore, in the article, we only consider the case of the energy function as a first-order Total Variation. On the other hand, the model (1) is a general form to solve the image restoration problem. For the SnP denoising problem, pixels of the noisy image can be classified: unchangeable pixels, salt pixels and pepper pixels and operator 𝐊 ≡ 𝐈. Therefore, the expression under the integral of the data fidelity term (i.e. the second term in the model (1)) can be rewritten as follows:

From (9) and (10), we can see that for the SnP denoising problem, the value 𝑝 = 1 is simpler than the cases of 𝑝 ≥ 2, because the corresponding Euler-Lagrange equation of the model (1) will not contain the data fidelity term (this matter will be const after taking derivative by 𝑢). Hence, the model (1) for the SnP denoising problem can be rewritten as follows: 𝑢 = argmin (∫ |∇𝑢|𝑑𝐱 + 𝜆 ∫ |𝑢 − 𝑣| 𝑑𝐱). 𝑢

Ω

(11)

Ω

For the norm of Total Variation, we have two types: 

Anisotropic Total Variation: |∇𝑢| = |𝑢𝑥1 | + |𝑢𝑥2 |



Isotropic Total Variation: |∇𝑢| = √𝑢𝑥21 + 𝑢𝑥22 .

Many works on denoising denoted that isotropic total variation is more effective than the anisotropic total variation. Therefore, in the article, we only consider the isotropic Total Variation. From all above mentioned, the explicit SnP denoising model based on Total Variation can be presented as:

𝑢 = argmin (∫ √𝑢2𝑥1 + 𝑢2𝑥2 𝑑𝐱 + 𝜆 ∫ |𝑢 − 𝑣| 𝑑𝐱). 𝑢

Ω

(12)

Ω

Selection of appropriate regularization parameter 𝜆 will improve denoising quality. The regularization parameter will be estimated based on characteristics of SnP noise. 2.3. Adaptive Regularization Parameter Estimation

ro

of

For natural images, gray values of pixels do not reach the boundary values 𝛿𝑚𝑎𝑥 , 𝛿𝑚𝑖𝑛 . So, if a gray value of a pixel equals to the boundary values, it can be considered as a noisy pixel. Especially, for high density SnP noise, the statement is more exact. In the case of denoising of real natural images, we propose method to estimate the regularization parameter 𝜆 as follows: 𝜇𝑣 (13) 𝜆 = min { , 1}, ℒ𝜂 where 𝜇𝑣 is the mean of corrupted pixels values with the noisy image 𝑣 normalized in the interval [0,1], ℒ𝜂 is the salt and pepper noise level. Let 𝐼, 𝐼𝑐 , 𝐼𝑚𝑎𝑥 , 𝐼𝑚𝑖𝑛 be a set of all pixels of an image, a set of corrupted pixels, a set of pixels with the gray value 𝛿𝑚𝑎𝑥 , a set of pixels with the gray value 𝛿𝑚𝑖𝑛 , respectively. In the case of SnP noise, we consider that 𝐼𝑐 = 𝐼𝑚𝑎𝑥 ∪ 𝐼𝑚𝑖𝑛 . For a noisy natural images with high density noise, the mean 𝜇𝑣 and the noise level 𝑞 can be evaluated as follows: ∑𝐼𝑐 𝑣𝐼𝑐 𝑐𝑎𝑟𝑑(𝐼𝑐 ) 𝜇𝑣 = , ℒ𝜂 = , 𝑐𝑎𝑟𝑑(𝐼𝑐 ) 𝑐𝑎𝑟𝑑(𝐼) where 𝑣𝐼𝑐 is gray values of pixels in 𝐼𝑐 , the notation 𝑐𝑎𝑟𝑑(∙) is set cardinality (the number of elements of a set). 2.4. Primal Dual Gradient Method for the Salt and Pepper Denoising

-p

We consider the optimization problem (12) in the discrete form as follows: 𝑛

𝑚

2

2

min {∑ ∑ (√(∇+𝑥 𝑢𝑖𝑗 ) + (∇+𝑦 𝑢𝑖𝑗 ) + 𝛼 + 𝜆|𝑢𝑖𝑗 − 𝑣𝑖𝑗 |)} , 𝑢

(14)

re

𝑖=1 𝑗=1

+ + + where ∇+ 𝑥 𝑢𝑖𝑗 = 𝑢𝑖+1,𝑗 − 𝑢𝑖𝑗 , ∇𝑦 𝑢𝑖𝑗 = 𝑢𝑖,𝑗+1 − 𝑢𝑖𝑗 , 0 < 𝛼 ≪ 1, (𝑖, 𝑗) ∈ {1, … , 𝑛} × {1, … , 𝑚}, ∇𝑥 𝑢𝑛,𝑖 = 0, ∇𝑦 𝑢𝑖,𝑚 = 0.

The problem (14) can be rewritten in the min-max problem:

𝑚

lP

𝑛

2

min max {∑ ∑ (√𝛼 (1 − |𝐩𝑖𝑗 | ) − 〈𝑢𝑖𝑗 , div 𝐩𝑖𝑗 〉 + 𝜆|𝑢𝑖𝑗 − 𝑣𝑖𝑗 |)} , 𝑢

|𝐩|∞ ≤1

(15)

𝑖=1 𝑗=1

2

2

− − − − − where 𝐩𝑖𝑗 = (∇− 𝑥 𝑝𝑖𝑗 , ∇𝑦 𝑝𝑖𝑗 ), |𝐩|∞ = max{|𝐩𝑖𝑗 |} , |𝐩𝑖𝑗 | = √(∇𝑥 𝑝𝑖𝑗 ) + (∇𝑦 𝑝𝑖𝑗 ) , ∇𝑥 𝑝𝑖𝑗 = 𝑝𝑖𝑗 − 𝑝𝑖−1,𝑗 , ∇𝑦 𝑝𝑖𝑗 = 𝑝𝑖𝑗 − 𝑝𝑖,𝑗−1 , 𝑖,𝑗

ur na

− − − ∇− 𝑥 𝑝1,𝑗 = −𝑝1,𝑗 , ∇𝑥 𝑝𝑛,𝑗 = 𝑝𝑛,𝑗 , ∇𝑦 𝑝𝑖,1 = −𝑝𝑖,1 , ∇𝑦 𝑝𝑖,𝑚 = 𝑝𝑖,𝑚 , div is a divergence operator, 〈∙,∙〉 is a scalar product, (𝑖, 𝑗) ∈ {1, … , 𝑛} × {1, … , 𝑚}.

The problem (15) contains two variables 𝑢 and 𝐩. 

Firstly, we solve the optimization problem for 𝑢 by fixing the dual variable 𝐩. The optimal condition is: 𝜆 𝑠𝑖𝑔𝑛(𝑢𝑖𝑗 − 𝑣𝑖𝑗 ) − div 𝐩𝑖𝑗 = 0.

(16)



Jo

We can solve (16) by the gradient descent method: [𝑘+1]

𝑢𝑖𝑗

[𝑘+1]

𝑢𝑖𝑗

[𝑘]

[𝑘]

[𝑘]

− 𝑢𝑖𝑗 = 𝜏 (𝜆 𝑠𝑖𝑔𝑛(𝑢𝑖𝑗 − 𝑣𝑖𝑗 ) − div 𝐩𝑖𝑗 ) , 𝜏 > 0 [𝑘]

[𝑘]

[𝑘]

= 𝑢𝑖𝑗 + 𝜏 (𝜆 𝑠𝑖𝑔𝑛(𝑢𝑖𝑗 − 𝑣𝑖𝑗 ) − div 𝐩𝑖𝑗 ).

Secondly, we solve the optimization problem for the dual variable 𝐩 with given 𝑢. The optimal condition is: 2

∇𝑢𝑖𝑗 √1 − |𝐩𝑖𝑗 | − √𝛼 𝐩𝑖𝑗 + 𝜕𝜙𝒟 (𝐩𝑖𝑗 ) = 0, where 𝜕 denotes for a subgradient, and 𝜙𝒟 (𝐩𝑖𝑗 ) = {

0, +∞,

𝑖𝑓 𝐩𝑖𝑗 ∈ 𝒟 , 𝑖𝑓 𝐩𝑖𝑗 ∉ 𝒟

𝒟 = {|𝐩𝑖𝑗 |∞ ≤ 1}.

(17)

We use the projection gradient method to solve (17): 2

[𝑘+1]

𝐩𝑖𝑗

[𝑘] [𝑘] [𝑘] [𝑘] 𝐩𝑖𝑗 + 𝜌 (∇𝑢𝑖𝑗 √1 − |𝐩𝑖𝑗 | − √𝛼 𝐩𝑖𝑗 )

= [𝑘] max {1, |𝐩𝑖𝑗

+

[𝑘] 𝜌 (∇𝑢𝑖𝑗 √1

−

[𝑘] |𝐩𝑖𝑗 |

2

, − √𝛼

[𝑘] 𝐩𝑖𝑗 )|}

where 𝜌 > 0. We can evaluate 𝑢[𝑘+1] , 𝐩[𝑘+1] with the initial conditions: 𝑢[0] = 𝑣, 𝐩[0] = 0. For the stop condition, we can use the number of iterations 𝑘 > 𝐾 or the tolerance |𝑢[𝑘] − 𝑢[𝑘−1] | ≤ 𝜖. Details of the proposed adaptive TV L1 regularization for SnP image denoising are presented in Algorithm 1. Algorithm 1. Adaptive Total Variation L1 Regularization for Salt and Pepper Image Denoising (ATVL1) Input: A noisy image 𝑣

of

Output: A denoised image 𝑢 Initialize: 𝑢[0] = 𝑣, 𝐩[0] = 0, 𝛼, 𝜌, 𝜏, 𝜖, 𝐾.

ro

Estimate λ = min{𝜇𝑣 ⁄ℒ𝜂 , 1}. For 𝑘 = 1,2,3, … , 𝐾

-p

For every pixel (𝑖, 𝑗) Compute: [𝑘]

[𝑘]

[𝑘]

= 𝑢𝑖𝑗 + 𝜏 (𝜆 𝑠𝑖𝑔𝑛(𝑢𝑖𝑗 − 𝑣𝑖𝑗 ) − div 𝐩𝑖𝑗 ) ,

re

[𝑘+1]

𝑢𝑖𝑗

2

= [𝑘] max {1, |𝐩𝑖𝑗

End

−

[𝑘] |𝐩𝑖𝑗 |

2

− √𝛼

. [𝑘] 𝐩𝑖𝑗 )|}

ur na

If |𝑢[𝑘] − 𝑢[𝑘−1] | ≤ 𝜖 Then

+

[𝑘] 𝜌 (∇𝑢𝑖𝑗 √1

lP

[𝑘+1]

𝐩𝑖𝑗

[𝑘] [𝑘] [𝑘] [𝑘] 𝐩𝑖𝑗 + 𝜌 (∇𝑢𝑖𝑗 √1 − |𝐩𝑖𝑗 | − √𝛼 𝐩𝑖𝑗 )

Return 𝑢.

End End

3.

Experimental Results and Discussion

Jo

We handle the experiments of the proposed SnP denoising method on MATLAB. The configuration of the computing system is Windows 10 Pro with Intel Core i5, 1.6GHz, 4GB 2295MHz DDR3 RAM memory. We configure the proposed adaptive TV-L1 based model and its primal dual gradient based implementation’s parameters as follows: 𝜖 = 𝛼 = 10−5 , 𝜏 = 0.02, 𝜌 = 6.25. Moreover, we limit the number of maximum iterations by 𝐾 = 500. 3.1. Denoising Quality Assessment To assess denoising quality, we use the full reference image quality metrics: The Peak Signal-to-Noise Ratio (PSNR) and the Structural Similarity (SSIM). The metrics are widely used in the image processing literature [13, 37, 38, 39, 40]. The PSNR [40] is defined as follows: 2 𝑢𝑚𝑎𝑥 𝑃𝑆𝑁𝑅 = 10 log10 ( ) 𝑑𝐵 𝑀𝑆𝐸 where 𝑚

𝑛

2 1 𝑀𝑆𝐸 = ∑ ∑ (𝑢𝑖𝑗 − 𝑢0𝑖𝑗 ) 𝑚𝑛 𝑖=1 𝑗=1

is the means square error with 𝑢0 is the original image (ground truth), 𝑢𝑚𝑎𝑥 denotes the maximum gray value, 𝑚 × 𝑛 is the image size, for e.g. for 8-bit images 𝑢𝑚𝑎𝑥 = 255. Note that higher PSNR (measured in decibels – dB) indicates better quality image. Structural similarity (SSIM) [40] is a better error metric for comparing the image quality than PSNR and the SSIM value is in the range of [0, 1] with value closer to one indicating better structure preservation. SSIM is computed between two images 𝜔1 , 𝜔2 with them same size of 𝑚 × 𝑛: (2𝜇𝜔1 𝜇𝜔2 + 𝑐1 )(2𝜎𝜔1𝜔2 + 𝑐2 ) 𝑆𝑆𝐼𝑀 = 2 , 2 + 𝑐 )(𝜎 2 + 𝜎 2 + 𝑐 ) (𝜇𝜔1 + 𝜇𝜔 1 𝜔1 𝜔2 2 2

where 𝜇𝜔𝑖 – the average of 𝜔𝑖 , 𝜎𝜔2𝑖 – the variance of 𝜔𝑖 , 𝜎𝜔1𝜔2 – the covariance, and 𝑐1 , 𝑐2 stabilization parameters. The parameters in the SSIM are set to the default values from [40] as follows: 𝑐1 = (𝐾1 𝐿)2 , 𝑐2 = (𝐾2 𝐿)2 , 𝐾1 = 0.01, 𝐾2 = 0.03; and 𝐿 = 255 for an 8-bit image. For image denoising, in the synthetic noise added case, the original noise-free and restored image is compared to find out how much structures are preserved by the restored method while removing noise. 3.2. Synthetic Images and Test Cases

re

-p

ro

of

We consider two test cases: denoise on a generated synthetic image and on real natural images of the UC-Berkeley dataset [30]. First test case: Denoising for the generated image. We create a synthetic image T01 as shown in Figure 1 with the black background (gray value is 0) containing a white disk (the gray value is 255), a white triangle (the gray value is 255) and a white rectangle (the gray value is 255) bounding a black disk (the gray value is 0). The image size is 256 × 256 pixels. Gray values of the edges pixels of the shapes are about 250 to 255.

Figure 1: The generated image with ID T01.

Noisy

MF

ACWMF

Wiener

Wavelet

Jo

Original

ur na

lP

We add SnP noise with the noise levels of 40% and 60% to the original image. Denoising results of the noise level of 40% is presented in Figure 2, and denoising results of the noise level of 60% is presented in Figure 3.

GSAP BPDF TVL1 ABF SSTV Proposed Figure 2: Denoising result of the proposed method for the noise level of 40% on the image T01. The PSNR/SSIM values are 6.99091/0.048548 (noisy), 15.8858/0.55127 (MF), 15.7249/0.54303 (ACWMF), 11.5905/0.087689 (Wiener), 12.7022/0.22318 (Wavelet), 23.5546/0.78349 (GSAP), 4.96828/0.13772 (BPDF), 27.2593/0.93534 (TVL1), 22.4554/0.8731 (ABF), 24.4298/0.6177 (SSTV), 27.5587/0.95022 (proposed).

For the noise level of 40%, Wiener filter worked ineffectively. A lot of noises still remain. Because the image only contains likenoisy gray values (0 and 255), BPDF failed. It destroyed all image structures. MF and ACWMF removed noise well but they caused some defects. Difference of denoising results of MF and ACWMF are very small. The wavelet denoising method removed noise well, but it caused many artifacts. Otherwise, edges of the shapes become too blurry. GSAP method removed noise effectively, but edges of the shapes are not preserved well. The edges are slightly blurry. Noise still remains in the result of ABF and the edges of various shapes in are not preserved well. SSTV model obtained similar edge artifacts along with staircasing associated with TV regularization. The

Noisy

MF

ACWMF

Wiener

Wavelet

-p

ro

Original

of

denoising results by TVL1 and the proposed method are the best. Our method is slightly better, but it is hard to differentiate by looking at the results visually. For this reason, we provide the corresponding optimal PSNR (dB) and SSIM values for each of the compared models, and the proposed method obtains the highest (i.e., the best denoising result) values in both the error metrics. For the noise level of 60%. The denoising results for Wiener filter, BPDF, MF, ACWMF, the wavelet denoising method, GSAP are similar with the case of 40% of noise. ABF cannot remove noise completely as can be seen from the homogenous regions inside the various shapes and did not preserve edges as well. SSTV model had strong staircasing artifacts and smearing of edges. Denoising results by our method and TVL1 are the best. In this case, our method preserved edges better than TVL1: clearly visible on the top angle of the triangle. Further, PSNR and SSIM values, show that denoising result by our method is the best.

re

GSAP BPDF TVL1 ABF SSTV Proposed Figure 3: Denoising result of the proposed method for the noise level of 60% on the image T01. The PSNR/SSIM values are: 5.22777/0.029176 (noisy), 9.85438/0.14092 (MF), 9.76191/0.13564 (ACWMF), 9.21434/0.060889 (Wiener), 9.92283/0.20164 (Wavelet), 7.34849/0.034765 (GSAP), 5.18428/0.1368 (BPDF), 23.2974/0.92804 (TVL1), 19.6484/0.6763 (ABF), 18.1944/0.3280 (SSTV), 23.4028/0.93505 (proposed).

Jo

ur na

lP

Second test case: Denoising for real natural images of the UC-Berkeley dataset. The dataset contains real images stored in the JPEG format. We select 20 images as shown in Figure 4 to implement the test.

Figure 4: The selected images and their IDs from the UC-Berkeley dataset (BSDS).

Figure 5 presents denoising results with various noise levels: 20%, 40%, 50%, 60%, 70% and 80% for the “man and woman” image with ID 157055. We can see that for the noise levels up to 50%, all details of the image are preserved well. For higher noise levels, some small details are lost, but the human shapes are preserved. For the very high noise-density such as 80%, the denoising result is acceptable (PSNR=19.2858, SSIM=0.49292). Although many details are lost, we still can discern the outlines of the man and the woman in the restored image.

Denoise of 20% PSNR/SSIM=25.3901/0.80403

40% noise PSNR/SSIM=9.03937/0.066233

Denoise of 40% PSNR/SSIM=23.9772/0.74241

50% noise PSNR/SSIM=8.04669/0.046153

Denoise of 50% PSNR/SSIM=23.2691/0.705

60% noise PSNR/SSIM=7.27292/0.033344

Denoise of 60% PSNR/SSIM=22.0877/0.63146

-p

ro

of

20% noise PSNR/SSIM=11.98/0.14154

Noisy

ur na

Original

lP

re

Denoise of 70% Denoise of 80% 70% noise 80% noise PSNR/SSIM=20.8733/0.56347 PSNR/SSIM=19.2858/0.49292 PSNR/SSIM=6.60379/0.023927 PSNR/SSIM=5.99761/0.014254 Figure 5: Denoising results of the proposed adaptive TV-L1 method for various noise levels on the “man and woman” image with ID 157055 from the BSDS.

Wavelet

ACWMF

GSAP

BPDF

Jo

Wiener

MF

TVL1 ABF SSTV Proposed Figure 6: Denoising results of the methods for the noise level of 60% on the plane image with ID 3096. PSNR/SSIM values are: 8.04419/0.0092163 (noisy), 12.7801/0.05909 (MF), 12.8155/0.060172 (ACWMF), 15.8154/0.065421 (Wiener), 21.3952/0.648 (Wavelet), 9.88408/0.013635 (GSAP), 30.5385/0.94996 (BPDF), 29.3758/0.94371 (TVL1), 31.7582/0.89253 (ABF), 29.3921/0.80429 (SSTV), 31.9951/0.95301 (proposed).

Figure 6 presents comparison of denoising results of the methods such as MF, ACWMF, Wiener filter, the wavelet denoising method, GSAP, BPDF, TVL1, ABF, SSTV and the proposed method for the noise level of 60%. As can be seen, the denoising results of MF and ACWMF still contains a lot of noise. Wiener filter, GSAP and the wavelet method work ineffectively. BPDF preserved details well, but it created the raindrop effect which manifests as streaking in plane’s contours. This can be clearly seen on the edges of the plane, and the raindrop effect will be stronger if we increase the noise level higher. Although ABF can preserve details well,

SnP noise still remains. SSTV removed noise well overall, however it created many staircasing artifacts, as can be seen in the cloud formations in the background as well as had smearing artifacts on the plane. For the denoising results of TVL1, the head of the plane is lost. Other details of the plane are not preserved well. The proposed method removed noise effectively with general structure preservation. Details are preserved well, except some small details such as the letter A on the tail of the plane which is lost. Overall, the denoising result by the proposed method is good, and the PSNR and SSIM values are highest for our proposed method. Table 1: The average PSNR and average SSIM values of denoising results of the methods with various noise levels

Wavelet GSAP BPDF TVL1 ABF SSTV Proposed

90%

PSNR

8.2805

6.8131

6.242

5.7279

SSIM

0.0392

0.0196

0.013

0.0078

PSNR

14.8

9.7494

7.9469

6.4588

SSIM

0.2353

0.0575

0.029

0.0133

PSNR

14.7892

9.7074

7.9115

6.4341

SSIM

0.2536

0.0624

0.0308

0.0136

PSNR

14.6791

13.1938

12.549

11.9015

SSIM

0.128

0.0837

PSNR

17.6991

15.6744

SSIM

0.4262

0.3711

PSNR

11.272

8.2565

SSIM

0.0637

0.0245

PSNR

25.8809

21.8272

SSIM

0.8307

0.6722

PSNR

22.7887

SSIM

0.6155

PSNR

27.2027

SSIM

0.8734

PSNR

24.4523

SSIM

0.649

PSNR

24.8968

SSIM

0.7033

of

Wiener

80%

0.0648

0.0484

14.8384

14.0001

0.3464

0.3211

7.4209

6.7294

0.0158

0.0093

18.3523

12.967

0.5321

0.3273

21.2474

20.2058

18.587

0.5662

0.5369

0.5034

21.9315

18.6648

15.0352

0.6049

0.3658

0.1602

20.9326

17.6521

14.3396

0.4894

0.3531

0.2129

22.2853

20.8486

18.8789

0.5962

0.5499

0.5069

ro

ACWMF

70%

-p

MF

50%

re

Noisy

Metrics

lP

Method

Conclusion

Jo

4.

ur na

Table 1 presents the average PSNR and average SSIM values of denoising results of the methods with the noise levels of 50%, 70%, 80% and 90%. For the noise level of 50%, the proposed method follows only BPDF and ABF by both PSNR and SSIM metrics. For the noise level of 70%, the average PSNR value of the denoising results of the proposed method is higher than ones of BPDF and ABF, but the average SSIM value is still lower. From the noise levels of 80% and 90%, the average PSNR value and the average SSIM value of the denoising results by our method are the highest. In particular, since our model is based on total variation with adaptive parameter estimation, we note the improvement obtained with the prior TVL1 in the Table 1 quantitatively. The proposed adaptive TV denoising method outperforms TVL1 in both PSNR and SSIM values at high SnP noise levels and this is confirmed by visually in Figure 6 on a real natural image. Despite the good results obtained we believe further improvements can be obtained by carefully selecting the regularization parameter in a multiscale manner [5] which can help us retain smaller textures (see Figure 6, front engines). This texture preservation is important in other image processing, and deep learning classification and retrieval tasks [6] where the image denoising is an integral part.

In the article, we have proposed an adaptive Total Variation salt and pepper denoising method. Our method based on the first-order Total Variation, L1 norm of data fidelity and adaptive regularization parameter estimation. The denoising algorithm is implemented based on the primal dual gradient method. From a variety of denoising results with various noise levels, we confirmed that, the proposed method removes noise effectively in synthetic and real natural images. For low-density and medium-density noises, all details are preserved well. For high-density noise, some details are lost, however main structures of the images are still preserved well in our proposed model compared to some of the recent related representative methods from median filters, Wiener filter, wavelets denoising, adaptive bilateral filter and regularization areas. The proposed method can be extended to denoise on color images. Extending to color and multispectral SnP denoising and adding a texture preserving component in our adaptive regularization parameter estimation are our current works in this direction.

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Conflict of Interest The authors declare no conflict of interest.

References [1] J. Shim, M. Yoon and Y. Lee, "Feasibility of newly designed fast non local means (FNLM)-based noise reduction filter for Xray imaging: A simulation study," Optik, vol. 160, pp. 124-130, 2018.

of

[2] R. Rojas and P. Rodriguez, "Spatially Adaptive Total Variation Image Denoising Under Salt and Pepper noise," in 19th European Signal Processing Conference EUSIPCO, pp 278-282, Barcelona, 2011.

ro

[3] D. N. H. Thanh and S. Dvoenko, "A denoising of biomedical images," The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XL, no. 5, pp. 73-78, 2015. [4] V. B. S. Prasath, D. N. H. Thanh and N. H. Hai, "Regularization Parameter Selection in Image Restoration with Inverse Gradient: Single Scale or Multiscale," in IEEE 7th International Conference on Communications and Electronics, pp. 278-282, Hue, 2018.

-p

[5] D. N. H. Thanh, V. B. S. Prasath, N. V. Son and L. M. Hieu, "An Adaptive Image Inpainting Method Based on the Modified Mumford-Shah Model and Multiscale Parameter Estimation," Computer Optics, vol. 43, no. 2, pp. 251-257, 2019. [6] T. F. Chan and J. Shen, Image Processing and Analysis: Variational, PDE, Wavelet and Stochastic Methods, SIAM, 2005.

re

[7] D. N. H. Thanh, E. Uğur, V. B. S. Prasath, V. Kumar and N. N. Hien, "A Skin Lesion Segmentation Method for Dermoscopic Images Based on Adaptive Thresholding with Normalization of Color Models," in IEEE 2019 6th International Conference on Electrical and Electronics Engineering, Istanbul, 2019.

lP

[8] V. Naghashi, "Co-occurrence of adjacent sparse local ternary patterns: A feature descriptor for texture and face image retrieval," Optik, vol. 157, p. 877–889, 2018. [9] V. B. S. Prasath and D. N. H. Thanh, "Structure tensor adaptive total variation for image restoration," Turkish Journal of Electrical Engineering & Computer Sciences, vol. 27, no. 2, pp. 1147-1156, 2019.

ur na

[10] D. N. H. Thanh, V. B. S. Prasath and L. M. Hieu, "A Review on CT and X-Ray Images Denoising Methods," Informatica, vol. 43, no. 2, pp. 151-159, 2019. [11] K. A. Hemant and M. Angshul, "Generalized Synthesis and Analysis Prior Algorithms with Application to Impulse Denoising," in Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image Processing, Bangalore, 2014. [12] U. Erkan and L. Gokrem, "A new method based on pixel density in salt and pepper noise removal," Turkish Journal of Electrical Engineering & Computer Sciences, vol. 26, pp. 162-171, 2018. [13] D. N. H. Thanh and S. Dvoenko, "A method of total variation to remove the mixed Poisson-Gaussian noise," Pattern Recognition and Image Analysis, vol. 26, no. 2, pp. 285-293, 2016.

Jo

[14] D. N. H. Thanh, V. B. S. Prasath and L. T. Thanh, "Total Variation L1 Fidelity Salt-and-Pepper Denoising with Adaptive Regularization Parameter," in IEEE 5th Nafosted Conference on Information and Computer Science NCIS'18, Ho Chi Minh city, 2018. [15] J. S. Lim, Two-dimensional signal and image processing, New Jersey: Prentice-Hall, 1990. [16] H. Hwang and R. A. Haddad, "Adaptive median filters: New algorithms and results," IEEE Transaction on Image Processing, vol. 4, no. 4, p. 499–502, 1995. [17] T. Chen and H. R. Wu, "Adaptive impulse detection using center-weighted median filters," IEEE Signal Processing Letters, vol. 8, no. 1, pp. 1-3, 2001. [18] F. Jin, P. Fieguth, L. Winger and E. Jernigan, "Adaptive Wiener filtering of noisy images and image sequences," in Proceedings 2003 International Conference on Image Processing, Barcelona, 2003. [19] X. Zhang, "Image denoising using local Wiener filter and its method noise," Optik, vol. 127, p. 6821–6828, 2016. [20] C. Tomasi and R. Manduchi, "Bilateral Filtering for Gray and Color Images," in IEEE The International Conference on Computer Vision (ICCV'98), Bombay, 1998.

[21] S. V. Priya and R. Seshasayanan, "Impulse Noise Removal Using Adaptive Bilateral Filter with Robust Noise Detector," Journal of Computational and Theoretical Nanoscience, vol. 13, no. 12, pp. 9090-9097, 2016. [22] I. M. Johnstone and B. W. Silverman, "Needles and Straw in Haystacks: Empirical Bayes Estimates of Possibly Sparse Sequences," Annals of Statistics, vol. 32, no. 4, p. 1594–1649, 2004. [23] F. Abramovich, Y. Benjamini, D. L. Donoho and I. M. Johnstone, "Adapting to Unknown Sparsity by Controlling the False Discovery Rate," Annals of Statistics, vol. 34, no. 2, p. 584–653, 2006. [24] A. Antoniadis and G. Oppenheim, "Wavelets and Statistics," in Lecture Notes in Statistics, New York, Springer Verlag, 1995. [25] D. L. Donoho, "De-noising by Soft-Thresholding," IEEE Transactions on Information Theory, vol. 42, no. 3, p. 613–627, 1995. [26] A. K. Aggarwal and A. Majumdar, "Hyperspectral Image Denoising Using Spatio-Spectral Total Variation," IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 3, pp. 442-446, 2016. [27] T. F. Chan and S. Esedoglu, "Aspects of Total Variation Regularized L1 Function Approximation," SIAM Journal on Applied Mathematics, vol. 65, no. 5, p. 1817–1837, 2005.

of

[28] T. F. Chan, G. H. Golub and P. Mulet, "A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration," SIAM Journal on Scientific Computing, vol. 20, no. 6, p. 1964–1977, 1999. [29] Z. Zhi, B. Shi and Y. Sun, "Primal-dual method to smoothing TV-based model for image denoising," Journal of Algorithms & Computational Technology, vol. 10, no. 4, p. 235–243, 2016.

ro

[30] P. Arbelaez, C. Fowlkes and D. Martin, "The Berkeley Segmentation," UC-Berkeley, [Online]. Available: https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/. [Accessed 10 12 2018].

-p

[31] V. B. S. Prasath, D. Vorotnikov, R. Pelapur, S. Jose, G. Seetharaman and K. Palaniappan, "Multiscale Tikhonov-Total Variation Image Restoration Using Spatially Varying Edge Coherence Exponent," IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5220-5235, 2015.

re

[32] L. Rudin, S. Osher and E. Fatemi, "Nonlinear total variation based noise removal algorithms," Physica D, vol. 60, no. 1-4, pp. 259-268, 1990. [33] H. Hu and J. Froment, "Nonlocal Total Variation for Image Denoising," in IEEE 2012 Symposium on Photonics and Optoelectronics, Shanghai, 2012.

lP

[34] W. Lu, J. Duan, Z. Qiu, Z. Pan, R. W. Liu and L. Bai, "Implementation of high‐ order variational models made easy for image processing," Mathematical Methods in the Applied Sciences, vol. 39, no. 14, pp. 4208-4233, 2016. [35] D. Li, X. Tian, Q. Jin and K. Hirasawa, "Adaptive fractional-order total variation image restoration with split Bregman iteration," ISA Transactions, vol. 82, pp. 210-222, 2018.

ur na

[36] T. Ringholm, J. Lazicyz and C.-B. Schönlieb, "Variational image regularization with Euler’s elastica using a discrete gradient scheme," SIAM Journal on Imaging Sciences, vol. 11, no. 4, p. 2665–2691, 2018. [37] V. B. S. Prasath, D. N. H. Thanh and N. H. Hai, "On Selecting the Appropriate Scale in Image Selective Smoothing by Nonlinear Diffusion," in IEEE 7th International Conference of Communications and Electronics, pp. 267-272, Hue, 2018. [38] V. B. S. Prasath, D. N. H. Thanh, N. H. Hai and N. X. Cuong, "Image Restoration With Total Variation and Iterative Regularization Parameter Estimation," in The Eighth International Symposium on Information and Communication Technology SoICT, pp. 378-384, NhaTrang, 2017.

Jo

[39] D. N. H. Thanh, S. Dvoenko and D. V. Sang, "Colour image denoising based on a combined model," in Seventh Symposium on Information and Communication Technology, pp. 308-315, Ho Chi Minh city, 2016. [40] Z. Wang, A. Bovik, H. Sheikh and E. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.