Robust infrared spectral deconvolution for image segmentation with spatial information regularization

Infrared Physics and Technology 102 (2019) 103011

Contents lists available at ScienceDirect

Infrared Physics & Technology journal homepage: www.elsevier.com/locate/infrared

Robust infrared spectral deconvolution for image segmentation with spatial information regularization Guangpu Shao, Tianjiang Wang

T

⁎

School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China

A R T I C LE I N FO

A B S T R A C T

Keywords: Resolution enhancement Infrared spectrometers Deconvolution Image segmentation

Spectral distortion often occurs in spectral data due to the influence of the point spread function of the infrared spectrometer. However, the infrared spectrum commonly exists the issues of random noises and band overlap. Resolution enhancement is usually the first step in the preprocessing procedure of material classification and segmentation. In this article, we have developed a resolution- enhancement algorithm with total variation (TVnorm) constraints for the degraded IR spectrum due to overlap and noises degradation. The point spread function is calculated according to infrared spectrometer system and Fourier-optical theory. Introducing the adaptive total variation, the constraint regularization, the proposed model can not only remove noises well but also estimate the point spread function simultaneously. This model is examined by a set of simulated IR spectrum with Poisson noises and a series of real IR spectra. Performance comparison with other state-of-the-art methods is made. The enhanced infrared spectrum can provide highly useful information for image segmentation.

1. Introduction

1.1. Related work

Image segmentation is the basic premise in vision analysis and material classification, and it has been a challenging problem, due to the texture content of the RGB images. The traditional RGB camera in material classification can only capture the shape and color information of material for image segmentation [1–8], but the infrared spectrometer sensing can capture the material component information, which are highly useful for accurately image segmentation. Fourier-transform infrared (IR) spectroscopy is widely utilized in the fields of materials recognition [9–14], modern defense [15–18], medical drug analysis [10,19–22] and biological materials [23–27]. IR spectrum signal could provide it “fingerprint” detail information and show the component on a molecular level [15,28] for material segmentation (Fig. 1). However, the IR spectrometer is usually limited by overlapped bands and random noises, tending to reduce the resolution of IR spectrum [29–31]. The spectral quality is also impacted by the working pressure and temperature in the factory. Combination of those factors, the observed IR spectrum lines are easy to misinterpretation, influencing the recognition accuracy of material classification and segmentation. Thus, it is important to remove the random noises in IR imaging spectrum before the succeeding segmentation processing [32].

In the past decade, many spectral deconvolution methods are developed to tackle the issue of Poissonian IR spectral degradation in material classification and segmentation [33–43]. Concerning these researches, the spectral deconvolution algorithms could be summed as two groups: analysis prior based method (ANAM) and synthesis prior based method (SYNM). For the ANAM, Kondo et al. proposed a Wiener filtering [44,45] method, which is a famous spectral deconvolution approaches. Firstly, Slima et al. [46,47] developed the Kalman-filter to raise IR spectral resolution. It has been widely used till today [33,48,49]. Senga et al. [50] proposed a homomorphic-filter-based method to raise the spectral resolution of the IR spectrum. Another classic Fourier-self deconvolution method (FSD) is proposed. FSD method becomes an important method applied in IR image segmentation field. For all the IR resolution-enhancement tasks, the point spread function is often assumed as a fixed function in the FSD method. Therefore, it cannot be applied to the real IR spectral. To achieve the good performance, KatraSnik et al. [51,52] designed an adjustable acousto-optical-filter to raise the IR spectral resolution. It can remove the spectral noises in the infrared spectrum data. However, it failed to process the infrared spectrum with strong noises. Recently, Wiener estimation [53,54] approach has been developed for spectra line with heavy noises, which can recover the missed structure as many as

⁎

Corresponding author. E-mail address: [email protected] (T. Wang).

https://doi.org/10.1016/j.infrared.2019.103011 Received 31 July 2019; Received in revised form 8 August 2019; Accepted 8 August 2019 Available online 16 August 2019 1350-4495/ © 2019 Elsevier B.V. All rights reserved.

Infrared Physics and Technology 102 (2019) 103011

G. Shao and T. Wang

(a)

0.8 0.6

Potatoes Soybeans

noise 0.4 area

(a

0.8

(b

0.6

0.2

0.4

0 2000

3000

1000 -1

(c) 500

550

600

650 750

structrue area

plain area

noise -free

noise

1000

0.2 0 450

(b)

Band overlap

2000

3000

Fig. 2. Analysis of high-resolution infrared spectrum data. (a) low-resolution IR spectrum. (b) high-resolution infrared spectrum data.

-1

Fig. 1. Infrared spectrum technology can distinguish the potatoes and soybeans for the image segmentation task [1].

transform (temperature and pressure), and contaminated by the Poisson noise Poisson(•). Therefore, the infrared imaging process can be written as [23,25,51,61]

possible. For the SYNM method [55], Richardson-Lucy (RL) is considered as the most famous one. But this approach often amplifies the infrared spectral noise along the iteration numbers. To solve the issue, Lu et al. [56] introduced the total variation regularization to improve the IR spectral resolution [57]. In [39,58,59], a convex regularization with the spatially constraint was proposed to recover the infrared spectrum data. Then, to split the overlap bands, Liu et al. [41] proposed by employing the robust Tikhonov constraint, which is based on infrared spectrum structures to remove the noise in plain regions as well as conserved the spectral band structure information. Several algorithms based Ridgelet transform regularization were also developed to raise the spectral resolution [19,23,60]. But those approaches cannot recover the infrared spectrum with the random noise (Poisson type).

in which g(v) and f(v) represent the clean spectrum (Fig. 2(b)) and the degraded infrared spectrum (Fig. 2(a)), respectively. And the symbol ⊗ respects the convolution operation between the clean spectrum and point spread function. h(v) denotes the point spread function reflecting the overlapping degree in the infrared spectrum. The model (1) is very difficult computed precisely with the Poisson noise, since g(v) and h(v) are both unknown. Therefore, we adopt the classical maximum a posteriori theory. In this theory, the point spread function h(v) and latent spectrum g(v) could be estimated by maximizing their probability value and Bayes criterion,

1.2. Contribution of this paper

in which the p(g, h|f) denotes the posterior-probability-distribution. After the logarithmic transformation, it could be written as

f (v )= Poisson(g (v )⊗ h (v ))

(1)

p(g, h|f )= p(f |g, h )p(h )p(h )

(2)

L

In this article, we propose a robust spectral deconvolution model with spatial information regularization for materials segmentation applications. The proposed model can recover the high-resolution IR spectral lines of foreground and background without any prior knowledge. Two aspect contributions of this paper can summed as follows.

− log(p (f |g,

(1) Adaptive total variation is introduced for IR spectrum deconvolution task in the materials segmentation for the first time. Since the robust total variation constraint, the resolution of degradation IR spectrum can be well enhanced in the spectral deconvolution iteration. (2) Comparison experiment results show that the proposed method achieves good performance with less residual noises. The highquality infrared spectrum could raise the recognition rate in material segmentation task.

The p(f) in (2) presents the infrared spectrum-prior, which constrain the smoothness in the IR spectral signal. It usually includes the Gaussian Markov and Laplacian Markov prior (see Fig. 3(a)). Generally, the form of the Markov-prior could be constructed as,

h )) ∝

∑ ((g

⊗

k ) v − fv log(g

⊗

h ) v)

v

(3)

2.2. NSSP prior for clean infrared spectrum

p(g )=

1 α exp − ρ(|∇g (v )|) M1 2

{

}

(4)

in which M1 is a constant, and the |▽g(v)| is a spatial activity measure with wavenumber v. It can be illustrated by the first order difference ▽g(v)=(g(v + 1)-g(v))/2. Then, the potential function ρ(•) in (4) is often considered as Laplacian-Markov,

1.3. Organization of this paper

ρ (k ) = ||k||TV This paper is organized as follows: In Section 2, the proposed IR spectral deconvolution model for materials segmentation is proposed. Section 3 shows the optimization process based on Split Bregman iteration. The simulation experimental results and discussion on the effect of noise level and computation speed are demonstrated Section 4. Moreover, the material segmentation incorporating the resolution improvement methods are validated. Finally, it concludes the paper in Section 5.

(5)

The advantage of the utilization of the Laplacian Markov prior is that some high-frequency component (such as Possion noise) in the IR spectrum tends to be removed. Therefore, a novelty noise suppression and structure preserving (NSSP) prior is designed in this paper. The potential function ρ(•) is constructed as the difference formulation in 0.04 0.03

2. IR-TVC models

0.02

0 0

Infrared spectral deconvolution is a fundamental problem in the field of infrared spectrum imaging. The clean IR spectrum g is degraded by band overlap, since the spectrometer working environment maybe

t=103

plain area

NSSP prior

0.01

t=104 0.02 0.03

0.04

detail region

t=104

t=103

0.01

2.1. Infrared spectrum imaging model

noise area

ș=wt=10 0|k|TV (a) t=0 ș=wt=10 2|k|TV TV-norm ș=wt=10 3|k|TV ș=wt=10 4|k|TV t=102

0

0.01

0.02

0.03

0.04

Fig. 3. (a) Traditional total variation prior. (b) Adaptive total variation prior. 2

Infrared Physics and Technology 102 (2019) 103011

G. Shao and T. Wang

the NSSP-prior and Laplacian-Markov prior. The NSSP function could be defined as,

3. Optimization of proposed model

ρ (k ) = exp[−θ|k |4 ] × ||k||TV

Lots of optimization technology could be employed to minimize the proposed model (9). In this article, the alternating minimization (AM) algorithm [36,62,63] combined with the split-Bregman-iteration is introduced to compute the latent IR spectrum. The optimization detail could be illustrated as follows.

(6)

in which θ denotes a constant parameter. It could distinguish the plain region, noises region and structure region. The Laplacian-Markov prior is a special situation of the NSSP prior, when the parameter θ is approached to zero. The exp[ − θ|∇g |4 ] in (6) is denoted as wNSSP in the following. For plain region because w(|▽g|) value is close to one, which means a large total variation strength is enforced to these points, and then the noise will be suppressed. For structure region since w(|▽g|) is small and almost close to zero and weaken the total variation strength, so the spectral detail and structure will be preserved.

3.1. Optimizing the point spread function h(v) With a fixed IR spectrum gn, the model (9) can be written as the simplified model, L

∑ [g ⊗

J (g, h )=

h − f log(g ⊗ h )] + β||∇h ||2 (10)

v

2.3. Adaptive total variation-based spectral deconvolution model

The iteration of the point spread function h is formulated as

For the PSF, the prior-probability is proposed according to the spectrum line smoothness. Their gradient information is utilized to constrain the spectrum spatial smoothness. In this paper, to constrain the point spread function, we introduce the Gaussian distribution function. The probability function of the point spread function is designed as

β 1 p(h )= exp ⎧− ||∇h||2 ⎫ ⎨ ⎬ M2 ⎩ 2 ⎭

f ⎤⎫ hn + 1 = hn ⎧g n + 1(−v )⊗ ⎡ n + 1 ⎢ ⎨ g ⊗ hn ⎥ ⎣ ⎦⎬ ⎩ ⎭ ×

hn + 1 =

where β denotes a regularization parameter, which could control the smoothness of the point spread function. Substituting the (6), (9) and (10) into (3), after some manipulation, M1 and M2 could be dropped. The probability function can be rewritten by three expressions. The infrared spectral estimation problem in (3) can written as the minimization of the energy function,

hn + 1 L ∑v

(hn + 1) v

(12)

3.2. Optimizing the latent spectrum g(v) Then, with the fixed point spread function h(v), the latent IR spectrum can be updated as, L

(8)

Q(g )=

∑ [−f log(g ⊗

h )+ g ⊗ h] + α

wNSSP ||∇g ||TV

v

and

h) L

∑ [g

⊗

h − f log( g ⊗

h )] + αwNSSP

v

||∇g

||TV + β|| ∇h

||2

(13)

It is very difficult to minimize this sub-problem since the item ||▽g||TV is non-smooth and non-convex. To convert (13) into an unconstrained problem, some auxiliary variables are introduced in the Bregman iterations. Then, the sub-problem could be written as the equation,

Q(g,

=

(11)

and

(7)

< g, h > = arg min[Q(g, h )]

1 {1 − β∇2 g }

L

Q(g )= ∑v [b1 − f log(b1)] + α

(9)

where α and β represent the balance weight values. Three items in the proposed model (9) could be described as follows:

such

that

wσ ||b2 ||TV b1 = g ⊗ h ,

b 2 = ∇g (14)

(1) The first item denotes the infrared spectrum data item, i.e., the spectral deconvolution should be meet as the observation model (1). (2) The second item enforces the NSSP regularization on the smoothness of the IR spectrum, since the NSSP function can split three kinds spectral regions (plain, noises and structure regions). Namely, the proposed potential function can adaptively tune the noise suppression strength with the aid of the spectral gradient feature. (3) The third item constrains the L2-norm on the point spread function intensity, so as to suppress the Poisson noise as well as preserve the PSF structures. (4) The aim of the article is to suppress the Poisson noise of the infrared spectrum, while saving the structure region and plain region of the latent spectrum as the degradation one; meanwhile, the point spread function need retain its smoothness. For simplicity's sake, the developed model is called as IR spectral deconvolution method via total variation constraints regularization (IR-TVC) for material segmentation.

Moreover, with the Bregman variables b1 and b2, the (14) could be written as L

Q(g )= ∑v [b1 − f log(b1)] + λ1 1 + 2γ

wNSSP ||b2 ||TV

{||d1 + g ⊗ h − b1 ||2 + ||d2 + ∇g

− b2 ||2 }

(15)

in which γ is Bregman parameter. Three sub-problems could be transformed from the (15) optimization problem, 1

||d1n + g ⊗ h − d1n ||2 + ||d 2n + ∇g − b2n ||2 } ⎧ g n + 1 = argmin 2γ { g ⎪ ⎪ b n + 1 = argmin ∑L [b − f log(b )] + 1 ||b n + g n + 1 ⊗ h − d ||2 1 1 1 1 v 2γ ⎪ 1 b1 1

n+1 ⎨ b2 = argminα wNSSP ||b2 ||1 + 2γ ||d2 + ∇h − b2 ||2 b2 ⎪ ⎪ d1n + 1 = d1n + (g n + 1 ⊗ h − b1n + 1) ⎪ n+1 ⎪ d 2 = d 2n + (∇g n + 1 − b2n + 1) ⎩

(16) in which n dentoes the iteration number. The first g-subproblem could 3

Infrared Physics and Technology 102 (2019) 103011

G. Shao and T. Wang

be computed by the follow equation,

(hT

⊗ h+

2I )g n + 1

=

hT

⊗

(d1n

−

b1n

0.8

)+

∇T (d 2n

−

b2n )

1 = (ηn + 2

in which

ηn

=

(ηn )2

d1n +1

b2n + 1 = max{||∇g n + 1 + d 2n + 1 ||αγ × wNSSP ,

(18)

b2n + 1 0}

∇g n + 1 + d 2n ||∇g n + 1 + d 2n ||2

0

20

(b)

40

Band overlap 3202

1696

3260

(d)

(c) Poisson noise

Band overlap

0.4

(19)

Poisson noise

0.2 0.0 1750

2250

2750

3250

-1

1750

2250

2750

3250 -1

Fig. 4. Simulated experiment for IR spectral data. (a) IR spectrum of Methyl Formate from 1550 to 3550 cm−1. (b) Noise-free infrared spectrum. (c) Noisy infrared spectrum. (d) Random noise (SNR = 200).

models were adopted, namely, ISD-WE, IRFAN and REφHS approaches. In Fig. 5(a)–(c), it demonstrates the deconvolution results in a simulated case using IRFAN, REφHS and IR-TVC approaches. The parameters are set as α = 0.05, β = 250 and 25 iterations. By the comparison of Fig. 5(d) with Fig. 5(a)– (c), it could observe that the IR-TVC can preserve the spectral structure meanwhile perform well in reducing the noises. Fig. 5(a)–(c) show that the peaks at 1979 cm−1 are lower than the ground-truth one, but the Fig. 5(d) the recovered peak is close to the groundtruth. And also, the degraded peak at 1696 cm−1 (in Fig. 4(b)) is reconstructed very well in Fig. 5(d). Other results exist the details missing phenomenon. Furthermore, we also investigate the peak distortions by comparing the spectral deconvolution performance of four methods. In Fig. 4(a), four peaks at 3260, 3209, 1979, and 1685 cm−1 are chosen for comparison. In Table 1, the peaks distortions are listed in their height and positions between the original spectrum and the deconvolution results. The RMSE values of the peaks distortions are calculated. It can be observed that the distortions of peak height and position by IR-TVC model demonstrate less in value than those by ISD-WE, IRFAN and REφHS models. It represents the restored spectrum in Fig. 5(d) is the closest to the ground-truth. It can conclude that the IR-TVC method can enhance the spectral resolution well when produce the spectral structure details.

4.1. Experimental setting In the simulated experiments, all the results of the proposed method are compared with three state-of-the-art approaches, namely, infrared spectral deconvolution with Winer estimation method (ISD-WE) [42], infrared spectral enhancement based on the fuzzy radial basis function neural networks (IRFAN) method [64], and spectral resolution-enhancement method with improved φHS regularization (REφHS) [39]. For examining its convergence, we record the normalized step difference energy (NSDE) along the iteration number. To evaluate the recovered infrared spectrum, we introduce two indexes, i.e., ratio of noise suppression (RNS) and ratio of the full width at half-maximum (RFWHM) [2,51,65], (20)

4.3. Effect of noises level

L

∑

-20

3209

0.6

To illustrate the effectiveness of our IR-TVC model, we conduct both simulation and real IR spectral deconvolution experiments on different material segmentation so as to compare the developed model with the state-of-the-art methods.

RFWHM

-40

0.8

could be optimized,

4. Experiments and discussions

1 = L

0.01

0.2

In this paper, we defined the convergence of the optimization process. Namely, both the infrared spectrum g(v) and point spread function h(v) are adjusted less than the preset constant values: ||hn+1-hn||/ ||hn|| < ε1 and ||gn+1-gn||/||gn|| < ε2. The preset values are set between 10−8 and 10−6. In our experiments, the height of all the IR spectrum is normalized to one.

∑ |∇f |/|∇g|

1981

0.02

0

0.4 1685

3.3. Numerical analysis and details

RNS =

PSF

0.0

+ 4αf )

+ h ⊗ f - α . Similarly,

1979

0.6

(17)

It is known that the (17) exists a closed-form solution. The optimization process of b1-related subproblem can be written as,

b1n + 1

(a)

FWHM(fv) /FWHM(gv)

v

FWHM(gv)

Afterward, to verify the robustness of spectral deconvolution methods to Poisson noises. The noisy spectrum is simulated in the Fig. 4(c) with noise level SNR = 200. In Fig. 6(a)-6(d), we illustrate the deconvolution results by ISD-WE, IRFAN, REφHS and IR-TVC, respectively. In a whole, the spectral noises (Fig. 4(d)) are removed well and recovered as 1blue spectrum in Fig. 6. Specifically, in Fig. 6(a), we can recognize the recovered spectrum presents less split while exists more residual noises. The fake peaks are indicated by the red arrow. Furthermore, the spectral structures are not reconstructed well (Fig. 6(b) and (c)) and missed some important details (band at 1685 cm−1). Shown in Fig. 5(d), the developed approach produces a very sharp spectrum. It is very similar with groundtruth one. The reason is the proposed the NSSP prior can distinguish the structure region, flat region and noises region in the degraded spectrum. But, the compared algorithms only consider the noise suppression ability. Furthermore, Table 2 reports the deconvolution results by four compared methods on the simulated infrared spectrum. As can be seen,

(21)

FWHM(v) f

in which the and represent the bands-width of the infrared spectrum f and g. The NSDE works robustly in our spectral deconvolution experiments, because the reference spectrum g is needed in the NSDE. Both the RNS and RFWHM, needing no-reference measure for infrared spectrum, are suitable for the real infrared spectrum. Moreover, RNS and RFWHM are capable of reflecting the overlap peaks splitting and noise removal. The larger the values of the two merits, the better the deconvoluted IR spectra results are. 4.2. Simulated experiment for noise-free case The IR spectrum is chosen from the spectral dataset of public network. The Gaussian-shape function was convoluted to the clean spectrum with the standard variance equaling to 8 cm−1 (see Fig. 4(a)). The degraded spectrum demonstrates more smoothness and a lower resolution, with the lower and wider peaks (see Fig. 4(c)). For example, the original bands of 3091, 3209 and 3260 cm−1 in Fig. 4(a) are overlap as one band. In Fig. 4(d), the overlap spectrum was contaminated by Poisson noise. To make comparison, three state-of-the-art

1 For interpretation of color in Fig. 6, the reader is referred to the web version of this article.

4

Infrared Physics and Technology 102 (2019) 103011

G. Shao and T. Wang

1981

(a)

0.2

3208

0 1980

0.8

(c)

0

1684

0.4

1750

3211

0.6 0.4

3206

2750

3250

4.4. Real infrared spectrum experiment

0.2 0 0.8

1979

0.4 0

We also applied the developed method on real infrared spectra data. IR spectrum often suffers from the Poisson noises problem, since the photonlimited detection in the material segmentation. Five infrared spectra achieved from [66] are verified. To save space, only two of them are demonstrated. For IR-TVC method, we set α = 0.05 and β = 300, initializing the point spread function by a Gaussian function with the standard deviation σ = 0.5 cm−1. Fig. 7(a) shows a 400 cm−1 length spectrum of Cr:LisAF crystal ranging from 301 to 699 cm−1. The overlapped peaks in Fig. 7(a) are deconvoluted into two peaks at 394 and 407 cm−1 in Fig. 7(b) (IRFAN method), 406 and 395 cm−1 in Fig. 7(d) (IR-TVC method). In Fig. 7(c), those bands are not split enough. At the aspect of noises suppression, the Fig. 7(d) obtain the

(d) 3209

0.2 2250

the proposed method achieves better performance than other algorithms.

(b)

0.6

0.6 0.2

1981

1685

0.4

1690

0.6

0.8

1693

0.8

3263

3090

1750

2250

2750

-1

3250 -1

Fig. 5. Simulation experiments without the Poisson noises. (a) ISD-WE [42] method. (b) IRFAN method [64]. (c) REφHS [39]. (d) Proposed IR-TVC model.

Table 1 Peak distortions of infrared spectrum recovered by ISD-WE, IRFAN, REφHS, and IR-TVC algorithms in Fig. 5. Band Param.

Methods

1435

1729

2841

2959

3010

RMSE

Position

ISD-WE IRFAN REφHS IR-TVC

+2 +3 −2 +1

−1 −3 +1 0

– – −1 +2

+2 +1 +2 0

+1 +2 +1 0

0.1857 0.1712 0.1569 0.0979

Height

ISD-WE IRFAN REφHS IR-TVC

−0.075 −0.112 −0.017 −0.016

−0.124 −0.143 −0.122 −0.102

– – −0.015 −0.013

−0.185 −0.178 −0.151 −0.012

−0.214 −0.182 −0.201 −0.010

1.6441 1.5171 1.3041 0.8369

(a) 0.8

1983

0.8

smooth infrared spectrum data. The IR spectrum of (D+)-glucopyranose [66] from 951 to 1199 cm−1 is shown in Fig. 8(a). In Fig. 8(b)–(d), we show the spectral deconvolution results of IRFAN, REφHS, and IR-TVC methods, respectively. The Poisson noise is well suppressed by all the infrared spectral deconvolution approaches. The overlapped band (indicated by left red arrow) is split to two peaks at 1067 and 1075 cm−1 in Fig. 8(d). However, this peak is still overlap in Fig. 8(b) and (c). Only two bands feature can be extracted in Fig. 7(c), while three bands can be extracted in Fig. 8(d). Spectral resolution is well recovered as well as the relative intensity distortion. In Table 3, the values of RNS and RFWHM are shown for the five IR spectra. All the spectral deconvolution approaches raise the RFWHM and RNS values, and the proposed method obtains the highest resolution.

(b)

1981

0.6

0.6

0.4

0.4 Residual noise

0.2

Ringing artifact

0.2

0

0 1983

0.8

1982

(c) 0.8

(d)

0.6

0.6 0.4

0.4 Residual noise

0.2

Less noise

0.2

0

0 1750

2250

2750

3250 -1

1750

2250

2750

3250 -1

Fig. 6. Noise suppression experiments for IR imaging spectrum (SNR = 200). (a) ISD-WE [42] model. (b) IRFAN model [64]. (c) REφHS [39] model. (d) Our IR-TVC model.

4.5. Discussions 1) Running speed. The proposed model achieves the high-resolution

Table 2 Comparison between the original spectrum and the recovered spectrum for noise-free and SNR = 200 (in brackets), in the index of NSDE, MCC, RFWHM and RNS. Materials

Merits

Spectral deconvolution algorithms by ISD-WE [42]

IRFAN [64]

REφHS [39]

IR-TVC

Methyl formate

NSDE MCC RFWHM RNS

0.0405 (0.0422) 0.9876 (0.9869) 1.95 (1.72) 1.42 (1.35)

0.0375 (0.0388) 0.9902 (0.9885) 2.63 (2.38) 1.89 (1.77)

0.0261 (0.0272) 0.9931 (0.9921) 2.99 (2.75) 2.03 (1.89)

0.0232 (0.0242) 0.9955 (0.994) 3.27 (2.99) 2.47 (2.25)

Tert-amyl alcohol

NSDE MCC RFWHM RNS

0.0352 (0.0382) 0.9887 (0.9881) 2.16 (1.95) 1.92 (1.82)

0.0313 (0.0331) 0.9929 (0.9917) 2.66 (2.31) 2.36 (2.04)

0.0256 (0.0278) 0.9935 (0.9926) 3.35 (3.08) 2.90 (2.67)

0.0225 (0.0246) 0.9948 (0.9942) 3.88 (3.65) 3.15 (2.98)

infrared spectrum via (9), accelerated by FFT. The Eqs (16)–(19) 5

Infrared Physics and Technology 102 (2019) 103011

G. Shao and T. Wang

0.8

(a)

Band overlap

0.6 Poisson noise 0.4

Sidelobe peak

561

0.05

(b)

542

407 394

0.04

0.2

0.03

0

(c)

542

2.0 1.5

(d)

542

557

0.5

441

0.02

406 395 Split

407 Overlap

1.0

556

5

442

Smoothness

600

500

400

600

500

-1

20

25

of the several state-of-the-art IR spectrum deconvolution methods. The running time lasts around 0.652 s for processing one infrared spectrum, which is faster than the REφHS method. 2) Convergence. The convergence property of the developed model is also tested. In Fig. 9, we present the NSDE and the number of iteration of one infrared spectrum of materials by the proposed method. It demonstrates the relationship between the NSDE value and the iteration number. It could be observed that the NSDE decreases dramatically along the iteration numbers, which verify the convergence (about 18 iterations) of the split Bregman iteration for optimizing the developed method.

-1

0.8

(a)

(b)

0.8 Band overlap 0.6

0.6 Poissian random noise

1075

1055 0.4 Ringing 0.2

0 0.8

(c) 0.8

1075 1054

0.4

(d)

1075 1067 1055

0.6

0.6

0.4

4.6. Applications in material segmentation Figs. 10-11 illustrate the results of the material segmentation with the aid of the infrared spectrum technology. We can see that the proposed IR-TVC method raised the resolution of the infrared spectrum. The higher resolution of the IR spectrum, the more accurately for material segmentation results. In Fig. 10, it shows the original image with potatoes and soybeans. It is very difficult to segment the two kinds food, because they have the very similar color and sharp. However, the infrared spectrum technology can distinguish them very well. They have different infrared spectrum lines. The proposed method can raise the resolution very well. Thus, it can achieve very well segmentation results for industrial materials and foods.

Smooth -ness

0.2

0.2

1000

1050

1100

1150 -1

1000

1050

1100

1150 -1

Fig. 8. Real IR spectrum deconvolution experiments. (a) Original infrared spectrum. (b) IRFAN result [64]. (c) REφHS result [39]. (d) Developed IR-TVC model.

Table 3 Values of RFWHM and RNS (in Bracket) by different spectral deconvolution approaches on five infrared spectral lines. Spectral lines

ISD-WE [42]

IRFAN [64]

REφHS [39]

IR-TVC

IR IR IR IR IR

3.27 2.64 2.91 3.19 2.82

3.75 2.85 3.27 3.66 3.22

4.16 3.28 3.67 3.91 3.63

4.56 3.72 4.12 3.35 4.05

1 2 3 4 5

15

400

Fig. 7. Real infrared spectrum experiment for the chemical material. (a) IR spectrum of (D + )-Glucopyranose [66] from 301 to 699 cm−1. (b) IRFAN result [64]. (c) REφHS result [39]. (d) Developed IR-TVC method.

0.2

10

Fig. 9. NSDE values along with the iteration number.

0

0.4

IRFAN R(ij+S IR-TVC

(2.11) (1.81) (2.09) (1.82) (1.91)

(2.32) (2.03) (2.31) (2.01) (2.16)

(2.59) (2.21) (2.62) (2.12) (2.47)

(2.98) (2.47) (2.81) (2.34) (2.73)

Table 4 Computation speed of different spectral deconvolution methods, such as the ISD-WE, IRFAN, REφHS and IR-TVC methods. Methods

Times

ISD-WE [42] IRFAN [64] REφHS [39] IR-TVC

8.356 s 2.254 s 3.726 s 0.652 s

could be calculated with the shrinkage operation parallelly. It is very fast and needs only several steps computations for each element. Therefore, all those rapid calculations make the developed method work in real-time. In Table 4, we compare the running time

Fig. 10. With the aid of the infrared spectrum technology, experiment results for the material segmented by [67]. (a) Original image with potatoes and soybeans. (b) Segmentation result without infrared spectrum. (c) With infrared spectrum recovered by REφHS [39]. (d) With infrared spectrum recovered by the proposed IR-TVC method. 6

Infrared Physics and Technology 102 (2019) 103011

G. Shao and T. Wang

[4]

[5] [6]

[7]

[8]

[9] [10]

[11]

[12]

Fig. 11. Material segmentation experiments with the aid of infrared spectrum technology. (a) Original images with different materials. Segmentation result by (b) With infrared spectrum recovered by IRFAN [64]. (c) With infrared spectrum recovered by REφHS [39]. (d) With infrared spectrum recovered by the proposed IR-TVC method.

[13]

[14] [15]

5. Conclusion [16]

In the paper, we have proposed a fast NSSP-based infrared spectrum resolution-enhancement algorithm for material segmentation task. Specifically, the proposed NSSP prior is utilized for differentiating the various structures (i.e. plain region, noises region and structure region) in the infrared spectrum. Besides using the adaptive total variation to describe the infrared spectrum features, we have also introduce the Tikhonov regularization for constraining the point spread function in the real infrared spectrum. A SBI-based optimization approach is developed to compute the high-resolution spectrum and point spread function. The IR spectral resolution enhancing experiments show the superior effect of the proposed method than some state-of-the-art approaches when we conducting visual comparison together with quantitative evaluation. Finally, the experiments on infrared spectrum datasets emphasize the effectiveness of our IR-TVC method. Considering the fast implementation, our IR-TVC method could be applied in solving various material segmentation.

[17] [18]

[19]

[20] [21]

[22]

[23]

[24]

Declaration of Competing Interest None

[25]

Acknowledgements The authors would like to thank Prof. Chen Shuo for their helpful discussion and offering the source Matlab codes. This work was supported by the Natural Science Foundation of China (Grant 61572214, Grant 61602244 and U1233119), Huazhong University of Science and Technology (No. 2016YXMS089) and Wuhan Science and Technology Bureau of Hubei Province, China (No. 2014010202-010110).

[26]

References

[29]

[27]

[28]

[30] [1] G. Shao, J. Gao, T. Wang, F. Liu, Y. Shu, Y. Yang, Image segmentation based on spatially coherent gaussian mixture model, International Conference on Digital Image Computing: Techniques and Applications (DICTA), 2014, pp. 1–6. [2] T. Liu, H. Liu, Z. Chen, A.M. Lesgold, Fast blind instrument function estimation method for industrial infrared spectrometers, IEEE Trans. Ind. Inf. 14 (2018) 5268–5277. [3] Z. Huang, H. Fang, Q. Li, Z. Li, T. Zhang, N. Sang, Y. Li, Optical remote sensing

[31]

[32] [33]

7

image enhancement with weak structure preservation via spatially adaptive gamma correction, Infrared Phys. Technol. 94 (2018) 38–47. H. Liu, Z. Zhang, S. Liu, J. Shu, T. Liu, T. Zhang, Blind spectrum reconstruction algorithm with L0-sparse representation, Meas. Sci. Technol. 26 (2015) 085502 (085501-085507). K. Sun, L. Liu, W. Tao, Progressive match expansion via coherent subspace constraint, Inf. Sci. 367–368 (2016) 848–861. H. Liu, Z. Zhang, S. Liu, J. Shu, T. Liu, Parametric spectral signal restoration via maximum entropy constraint and its application, 2015 IEEE Signal Processing and Signal Processing Education Workshop (SP/SPE), IEEE, Salt Lake City, Utah, USA, 2015, pp. 353–357. August 9-12. Z. Huang, L. Huang, Q. Li, T. Zhang, N. Sang, Framelet regularization for uneven intensity correction of color images with illumination and reflectance estimation, Neurocomputing 314 (2018) 154–168. H. Liu, L. Yan, Y. Chang, H. Fang, T. Zhang, Spectral deconvolution and feature extraction with robust adaptive Tikhonov regularization, IEEE Trans. Instrum. Meas. 62 (2013) 315–327. B. Pan, Z. Shi, X. Xu, Longwave infrared hyperspectral image classification via an ensemble method, Int. J. Remote Sens. 38 (2017) 6164–6178. Z. Huang, L. Chen, Y. Zhang, Z. Yu, H. Fang, T. Zhang, Robust contact-point detection from pantograph-catenary infrared images by employing horizontal-vertical enhancement operator, Infrared Phys. Technol. 101 (2019) 146–155. T. Liu, Y. Li, H. Liu, Z. Zhang, S. Liu, RISIR: rapid infrared spectral imaging restoration model for industrial material detection in intelligent video systems, IEEE Trans. Ind. Inf. (2019), https://doi.org/10.1109/TII.2019.2930463. G. Ycas, F.R. Giorgetta, E. Baumann, I. Coddington, D. Herman, S.A. Diddams, N.R. Newbury, High-coherence mid-infrared dual-comb spectroscopy spanning 2.6 to 5.2 μm, Nat. Photonics 12 (2018) 202–208. X. Wang, M.-Y. Zhai, Z.-H. Mao, Y.-F. Lu, J.-H. Yin, Fourier transform infrared spectroscopic imaging application for multi-stage discrimination in cartilage degeneration, Infrared Phys. Technol. 92 (2018) 122–127. H. Liu, S. Liu, Z. Zhang, J. Sun, J. Shu, Adaptive total variation-based spectral deconvolution with the split Bregman method, Appl. Opt. 53 (2014) 8240–8248. I. Mukhopadhyay, B.E. Billinghurst, High resolution synchrotron radiation Fourier transform infrared spectrum of the COH-bending mode in methanol-D1 CH2DOH, Infrared Phys. Technol. 85 (2017) 184–210. Y.-G. Zhang, X.-M. Shao, Y.-N. Zhang, Y. Gu, X.-Y. Chen, Y.-J. Ma, X. Li, H.-M. Gong, J.-X. Fang, Correction of FTIR acquired photodetector response spectra from midinfrared to visible bands using onsite measured instrument function, Infrared Phys. Technol. 92 (2018) 78–83. R. Zhao, Z. Shi, Z. Zou, Z. Zhang, Ensemble-based cascaded constrained energy minimization for hyperspectral target detection, Remote Sensing 11 (2019) 1310. Z. Huang, Y. Zhang, Q. Li, T. Zhang, N. Sang, H. Hong, Progressive dual-domain filter for enhancing and denoising optical remote-sensing images, IEEE Geosci. Remote Sens. Lett. 15 (2018) 759–763. Q. Liu, Z. Guo, S. Xiao, H. Yu, Depth IR spectroscopic data resolution improvement for antibiotics component analysis in critically ill elderly patients, Infrared Phys. Technol. 93 (2018) 291–299. H. Liu, Z. Zhang, J. Sun, S. Liu, Blind spectral deconvolution algorithm for Raman spectrum with Poisson noise, Photon. Res. 2 (2014) 168–171. Z. Huang, Y. Zhang, Q. Li, T. Zhang, N. Sang, Spatially adaptive denoising for X-ray cardiovascular angiogram images, Biomed. Signal Process. Control 40 (2018) 131–139. T. Liu, H. Liu, Y. Li, Z. Zhang, S. Liu, Efficient blind signal reconstruction with wavelet transforms regularization for educational robot infrared vision sensing, IEEE/ASME Trans. Mechatron. 24 (2019) 384–394. H. Liu, Y. Li, Z. Zhang, S. Liu, T. Liu, Blind Poissonian reconstruction algorithm via curvelet regularization for an FTIR spectrometer, Opt. Express 26 (2018) 22837–22856. M.J. Baker, J. Trevisan, P. Bassan, R. Bhargava, H.J. Butler, K.M. Dorling, P.R. Fielden, S.W. Fogarty, N.J. Fullwood, K.A. Heys, C. Hughes, P. Lasch, P.L. Martin-Hirsch, B. Obinaju, G.D. Sockalingum, J. Sulé-Suso, R.J. Strong, M.J. Walsh, B.R. Wood, P. Gardner, F.L. Martin, Using Fourier transform IR spectroscopy to analyze biological materials, Nat. Protoc. 9 (2014) 1771–1791. T. Liu, Z. Chen, H. Liu, Z. Zhang, FTIR spectral imaging enhancement for teacher’s facial expressions recognition in the intelligent learning environment, Infrared Phys. Technol. 93 (2018) 213–222. B. Yi, X. Shen, H. Liu, Z. Zhang, W. Zhang, S. Liu, N. Xiong, Deep matrix factorization with implicit feedback embedding for recommendation system, IEEE Trans. Ind. Inf. 15 (2019) 4591–4601. T. Liu, H. Liu, Y. Li, Z. Chen, Z. Zhang, S. Liu, Flexible FTIR spectral imaging enhancement for industrial robot infrared vision sensing, IEEE Trans. Ind. Inf. (2019), https://doi.org/10.1109/TII.2019.2934728. H. Liu, Z. Zhang, S. Liu, T. Liu, L. Yan, T. Zhang, Richardson-Lucy blind deconvolution of spectroscopic data with wavelet regularization, Appl. Opt. 54 (2015) 1770–1775. C.J. Manning, P.R. Griffiths, Noise sources in step-scan FT-IR spectrometry, Appl. Spectrosc. 51 (1997) 1092–1101. K. Sun, W. Tao, A constrained radial agglomerative clustering algorithm for efficient structure from motion, IEEE Signal Process Lett. 25 (2018). H. Liu, Z. Zhang, S. Liu, J. Shu, T. Liu, L. Yan, T. Zhang, Spectral blind deconvolution with differential entropy regularization for infrared spectrum, Infrared Phys. Technol. 71 (2015) 481–491. H. Liu, Z. Zhang, S. Liu, L. Yan, T. Liu, T. Zhang, Joint baseline-correction and denoising for raman spectra, Appl. Spectrosc. 69 (2015) 1013–1022. H. Liu, T. Zhang, L. Yan, H. Fang, Y. Chang, A MAP-based algorithm for

Infrared Physics and Technology 102 (2019) 103011

G. Shao and T. Wang

[50] Y. Senga, K. Minami, S. Kawata, S. Minami, Estimation of spectral slit width and blind deconvolution of spectroscopic data by homomorphic filtering, Appl. Opt. 23 (1984) 1601–1608. [51] J. KatraSnik, F. Pernu, B.t. Likar, Deconvolution in acousto-optical tunable filter spectrometry, Appl. Spectrosc. 64 (2010) 1265–1273. [52] H. Liu, S. Liu, T. Huang, Z. Zhang, Y. Hu, T. Zhang, Infrared spectrum blind deconvolution algorithm via learned dictionaries and sparse representation, Appl. Opt. 55 (2016) 2813–2818. [53] S. Chen, G. Wang, X. Cui, Q. Liu, Stepwise method based on Wiener estimation for spectral reconstruction in spectroscopic Raman imaging, Opt. Exp. 25 (2017) 1005–1018. [54] H. Liu, L. Yan, T. Huang, S. Liu, Z. Zhang, Blind spectral signal deconvolution with sparsity regularization: an iteratively reweighted least-squares solution, Circuits Syst Signal Process. 36 (2017) 435–446. [55] L. Yan, H. Liu, S. Zhong, H. Fang, Semi-blind spectral deconvolution with adaptive Tikhonov regularization, Appl. Spectrosc. 66 (2012) 1334–1346. [56] L. Deng, H. Zhu, G. Lu, THz spectrum deconvolution with Total variation regularization, 2013 6th International Congress on Image and Signal Processing (CISP), Hangzhou, China, 2013, pp. 1272–1276. [57] Zhenwei-Shi Shuo-Yang, Hyperspectral image target detection improvement based on total variation, IEEE Trans. Image Process 25 (2016) 2249–2258. [58] H. Zhu, L. Deng, H. Li, Y. Li, Deconvolution methods based on convex regularization for spectral resolution enhancement, Comput. Electr. Eng. 70 (2018) 959–967. [59] T. Liu, Z. Chen, S. Liu, Z. Zhang, J. Shu, Blind image restoration with sparse priori regularization for passive millimeter-wave images, J. Visual Commun. Image Represent 40 (Part A) (2016) 58–66. [60] K. Sun, P. Li, W. Tao, Y. Tang, Feature guided biased gaussian mixture model for image matching, Inf. Sci. 295 (2015) 323–336. [61] D.C. Fernandez, R. Bhargava, S.M. Hewitt, I.W. Levin, Infrared spectroscopic imaging for histopathologic recognition, Nat. Biotechnol. 23 (2005) 469. [62] K. Sun, L. Liu, W. Tao, Image matching via feature fusion and coherent constraint, IEEE Geosci. Remote Sens. Lett. 14 (2017) 289–293. [63] K. Sun, W. Tao, A center-driven image set partition algorithm for efficient structure from motion, Inf. Sci. 479 (2019) 101–115. [64] S.B. Roh, S.K. Oh, W. Pedrycz, Identification of black plastics based on fuzzy RBF neural networks: focused on data preprocessing techniques through Fourier transform infrared radiation, IEEE Trans. Ind. Inf. 14 (2018) 1802–1813. [65] T. Liu, H. Liu, Z. Zhang, S. Liu, Nonlocal low-rank-based blind deconvolution of Raman spectroscopy for automatic target recognition, Appl. Opt. 57 (2018) 6461–6469. [66] S.B.Engelson, Infrared Spectral of D(+)-Cellobiose, in, 2019. < http://www. models.life.ku.dk/specarb > . [67] G. Shao, J. Gao, T. Wang, F. Liu, Y. Shu, Y. Yang, Fuzzy c-means clustering with a new regularization term for image segmentation, International Joint Conference on Neural Networks (IJCNN), 2014, pp. 2862–2869.

spectroscopic semi-blind deconvolution, Analyst 137 (2012) 3862–3873. [34] W. Zhang, D. Wen, Z. Song, G. Liu, X. Wei, Z. Li, Spectrum reconstruction in Fourier transform imaging spectroscopy based on high-performance parallel computing, Appl. Opt. 57 (2018) 5983–5991. [35] V. Sizikov, D. Sidorov, Discrete spectrum reconstruction using integral approximation algorithm, Appl. Spectrosc. 71 (2017) 1640–1651. [36] T. Liu, H. Liu, Z. Chen, Y. Chen, S. Wang, Z. Liu, H. Zhang, FBRDLR: fast blind reconstruction approach with dictionary learning regularization for infrared microscopy spectra, Infrared Phys. Technol. 90 (2018) 101–109. [37] Q. Han, Q. Xie, S. Peng, B. Guo, Simultaneous spectrum fitting and baseline correction using sparse representation, Analyst 142 (2017) 2460–2468. [38] Z. Huang, Y. Zhang, Q. Li, Z. Li, T. Zhang, N. Sang, S. Xiong, Unidirectional variation and deep CNN denoiser priors for simultaneously destriping and denoising optical remote sensing images, Int. J. Remote Sens. 40 (2019) 5737–5748. [39] H. Zhu, L. Deng, G. Xu, Y. Chen, Y. Li, Spectral semi-blind deconvolution methods based on modified φHS regularizations, Opt. Laser Technol. 110 (2019) 24–29. [40] Z. Huang, Q. Li, T. Zhang, N. Sang, H. Hong, Iterative weighted sparse representation for X-ray cardiovascular angiogram image denoising over learned dictionary, IET Image Proc. 12 (2018) 254–261. [41] H. Liu, M. Zhou, Z. Zhang, J. Shu, T. Liu, T. Zhang, Multi-order blind deconvolution algorithm with adaptive Tikhonov regularization for infrared spectroscopic data, Infrared Phys. Technol. 71 (2015) 63–69. [42] S. Chen, X. Lin, C. Yuen, S. Padmanabhan, R.W. Beuerman, Q. Liu, Recovery of Raman spectra with low signal-to-noise ratio using Wiener estimation, Opt. Express 22 (2014) 12102–12114. [43] H. Liu, Z. Zhang, S. Liu, J. Shu, Z. Liu, Blind infrared spectroscopic data restoration with the similarity of multi-scales regularization, IEEE Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, Hong Kong, December, 2015, pp. 17–20. [44] K. Kondo, Y. Ichioka, T. Suzuki, Image restoration by Wiener filtering in the presence of signal-dependent noise, Appl. Opt. 16 (1977) 2554–2558. [45] H. Liu, Z. Zhang, S. Liu, T. Liu, Entropy Constrained-based Blind Deconvolution Approach for Overlap Band Narrowing, 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia, October 19-21, 2015, pp. 452–457. [46] M.B. Slima, R.Z. Morawski, A. Barwicz, Kalman-filter-based algorithms of spectrophotometric data correction III. use of splines for approximation of spectra, IEEE Trans. Instrum. Meas. 46 (1997) 685–689. [47] S. Chen, Kalman filter for robot vision: a survey, IEEE Trans. Ind. Electron. 59 (2012) 4409–4420. [48] F. Auger, M. Hilairet, J.M. Guerrero, E. Monmasson, T. Orlowska-Kowalska, S. Katsura, Industrial applications of the Kalman Filter: a review, IEEE Trans. Ind. Electron. 60 (2013) 5458–5471. [49] H. Liu, Z. Zhang, S. Liu, T. Liu, Y. Chang, Destriping algorithm with L0 sparsity prior for remote sensing images, IEEE International Conference on Image Processing (ICIP 2015), Quebec, Canada,, September, 2015, pp. 27469–30473.

8