- Email: [email protected]
Infrared image segmentation algorithm for defect detection based on FODPSO
Infrared Physics and Technology 102 (2019) 103051
Contents lists available at ScienceDirect
Infrared Physics & Technology journal homepage: www.elsevier.com/locate/infrared
Infrared image segmentation algorithm for defect detection based on FODPSO
T
⁎
Qingju Tang , Shuaishuai Gao, Yongjie Liu, Fengyun Yu School of Mechanical Engineering, Heilongjiang University of Science and Technology, Harbin 150022, PR China
A R T I C LE I N FO
A B S T R A C T
Keywords: Fractional order Fractional-order Darwin particle swarm algorithm Infrared image Edge detection Multi-scale segmentation Algorithm comparison SiC coating
To address the problems of high noise and fuzzy edges of acquired infrared images when detecting debonding defective components of SiC thermal barrier coating structure by pulsed infrared thermal waves, an image segmentation method based on fractional Darwin particle swarm optimization algorithm (FODPSO) is adopted to reconstruct infrared image sequences. This algorithm adopts fractional calculus to control the convergence of the system, and can perform n − 1 threshold optimization calculation on n − 1 level images. Experimental results show that FODPSO algorithm effectively overcomes the shortcomings of traditional algorithm such as easy to enter local optimal and slow convergence speed, and can realize the edge identification of SiC coating defects. The processed image improves the contrast and signal-to-noise ratio.
1. Introduction Infrared thermography (IT) is a safe non-destructive testing (NDT) technique that has a fast inspection rate and is generally contact-less. It is used for diagnostics and monitoring in several fields such as electrical components, thermal comfort, buildings, artworks, medical, security and composite materials [1], and in most of these applications wellknown infrared approaches have been utilized for thermal image enhancement, thermal image segmentation, and particularly defect segmentation in IRNDT [2]. In the process of detecting the structural defects of the pulsed infrared heat wave, the surface of the component is heated by the instantaneous heat pulse emitted by the high-energy flash lamp. It can be combined with imaging in order to discriminate between materials and/or attend to “determine a period of time” of execution for an object [3], and the image changes of the surface temperature field is recorded by the infrared camera. The thermal image sequence is analyzed and processed to observe internal defects or structural discontinuities in the component. The detection of defects in the original thermal image collected directly from the infrared camera is the main method of IR NDT – infrared thermographic nondestructive testing [4–6]. The difference between a visible image and an infrared image is that the visible image is a representation of the reflected light on the scene, whereas in the infrared image, the scene is the source and can be observed by an infrared camera without light. Images acquired using infrared cameras are converted into visible images by assigning a color to each infrared energy level [7]. The technique is based on
⁎
remote infrared detection of surface temperature anomalies that evolve in response to the application of a thermal perturbation to the structure. Relative to radiography and ultrasound, the most commonly applied nondestructive inspection techniques for composite materials, thermography offers significant gains in both ease and speed of inspection [8]. However, because the surface temperature caused by defects or structural anomalies with smaller diameters is very small compared to the background temperature, that is, the original thermal image is a typical low-contrast image, its ability to identify local abnormal changes inside the specimen is very limited. Segmentation is essential to identify the defects after applying the prepossessing techniques. This section briefly reviews the methods employed for segmentation [9]. Image segmentation is the main technique in image processing. The method is to separate the target from the background and provide a basis for subsequent classification, recognition and retrieval. Image segmentation applies one or more operations to divide the image into regions with similar characteristics. Image segmentation methods mainly include threshold methods and regional methods. The threshold method uses the gray frequency information to segment the distribution information, and the region method uses the local spatial information to perform segmentation. The image threshold method is the most used and simplest method. Optimization plays an important role in computer science, artificial intelligence, operational research, and other related fields. It is the process of trying to find the best possible solution to an optimization problem within a reasonable time limit [10]. The particle swarm optimization (PSO) algorithm is a stochastic global optimization
Corresponding author. E-mail address: [email protected] (Q. Tang).
https://doi.org/10.1016/j.infrared.2019.103051 Received 25 November 2018; Received in revised form 13 August 2019; Accepted 23 September 2019 Available online 23 September 2019 1350-4495/ © 2019 Elsevier B.V. All rights reserved.
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
calculation method based on swarm intelligence. The algorithm has the advantages of fast convergence and easy implementation. A particularly important capability is image segmentation. Segmentation is the process of partitioning an image into a set of non-intersecting regions, such that each region is homogeneous, but the union of no two adjacent regions is homogeneous. This is a fundamental problem in Computer Vision, and a number of methods have been proposed for solving it [11]. However, it also shows some problems to be improved in practical applications, that is, the algorithm tends to converge quickly in the early stage of search, but in the later stage, the particle group will tend to be the same (Loss of diversity), the convergence speed is obviously slower, the search accuracy is reduced, and the algorithm is easy to fall into local optimum. Therefore, many scholars are working to improve the performance of the PSO algorithm [12–14]. When performing the image segmentation, how to determine the number of categories of image segmentation that has always been a hot and difficult point of research. Tillett et al. [15] proposed a Darwinian particle swarm optimization (DPSO) algorithm that uses natural selection evolution to dynamically divide a population into subgroups. Each subgroup is searched independently to improve particle diversity and enhance the global optimization ability of the algorithm. Pires et al. [16] proposed FODPSO is an extensible DPSO algorithm, in which fractional order α is used to control the convergence speed of the algorithm. It has an important property: on the one hand, an integer order calculus represents a finite series. On the other hand, a fractional order calculus represents an infinite term series. Therefore, integer-order calculus is a local operator, and fractional-order calculus is a representation of all the states in the past. With the increase of time, the influence of the past states gradually weakens. Because of the inherent nature of fractional-order calculus, it can describe the irreversibility and chaos well, making the particle trajectory more suitable for the global optimal situation. The fitness values of initial individuals are calculated, the particle velocity updates and fitness values are calculated, and compared with the previous generation optimal fitness values, the optimal fitness of the population is updated until the number of iterations is reached. The FODPSO algorithm has better computational accuracy and convergence speed than PSO and DPSO (Darwinian Particle Swarm Optimization). The convergence performance of FODPSO algorithm depends on the fractional order directly. When α increasing, the convergence speed becomes slower, and when it decreases, the population falls into the local maximum. The probability of superiority becomes higher.
Fig. 1. FODPSO algorithm flow chart.
threshold scales t j , j = 1, 2, 3⋯,n − 1, the image after segmentation is as follows:
⎧ ⎪ ⎪ F (x , y ) =
2. n-Scale threshold segmentation model The robustness of n-scale threshold optimization is a difficult point for image segmentation. Common image gray values are 0–255. Set a maximum gray value L = 256, then for a gray image, the gray threshold can be taken as {0, 1, 2, …, L − 1}, correspondingly, pm , the probability of each gray value, could be expressed as [17,18].
pm =
hm , N
m−1
+ t2), t1 < f (x , y ) ⩽ t2
⋮ ⎨1 ⎪ 2 (tn − 2 + tn − 1), tn − 2 < f (x , y ) ⩽ tn − 1 ⎪ L, f (x , y ) > tn − 1 ⎩
(3)
where the function f (x , y ) represents the gray value corresponding to the image coordinate (x , y ) , x corresponds to the width W coordinate, y corresponds to the height H coordinate; F (x , y ) is the divided image gray value, and the image is divided into D1, D2 , ⋯Dn types. It can be known from Eq. (3) that the image can be segmented by finding n − 1 different threshold scales t j , and how to find the threshold evaluation function (adaptive function) becomes the next problem to be solved [12,13,17]. The simplest and fastest threshold optimization method is to directly find the gray value corresponding to the largest variance between different classes, which is generally defined as follows:
N
∑ pm
0, f (x , y ) ⩽ t1 1 (t 2 1
=1 (1)
where m is the specific image gray value, 0 ⩽ m ⩽ L − 1; N is the number of pixels in the image, N = H × W ; hm is the total number of gray values of the image, hm can be directly obtained from the statistical histogram; then the average value of the gray is:
n
σB2 =
L
μT =
∑ mpm m=1
∑ ωj (μj − σT )2 j=1
(2)
(4)
where j represents a concrete class, ωj represents the probability of the j class, μj is the mean of the j class, and ωj and μj of the specific D1, D2 , ⋯Dn class are expressed as follows [17]:
According to the calculated threshold μT , the image gray value is greater than or equal to μT divided into one category, and less than μT is divided into other category, which becomes a common image binarization process. Here, the two-dimensional threshold scale problem is extended to the n -dimensional threshold scale, that is n − 1 different 2
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
Flash lamp
Infrared Thermal Imager
Thermal wave signal Computer
Trigger signal
Trigger signal
Specimen Synchronous trigger Mask
Pulse power supply
Fig. 2. Diagram of optical pulsed infrared thermal wave nondestructive testing system.
μj =
⎧ ⎪ ⎪ ⎪
tj
∑ m=1 tj
mpm ωj
,j=1
mp
m ,1
∑
(6)
From Eqs. (4)–(6), Eqs. (7)–(9) are available: 2
⎛ t ⎜ 1 2 2 σB (t j = 1) = ωj (μj − μT ) = ∑ pm × ∑ ⎜ m=1 m=1 ⎜ ⎝ t1
⎛ ⎞ ⎜ mpm ⎟ − ⎜ t1 ⎟ ⎜ ∑ pm ⎟ ⎝ m=1 ⎠
L
∑ m=1
⎞ ⎟ (mpm ) ⎟ ⎟ ⎠ (7)
Fig. 3. Input infrared image processed by PCA.
σB2 (t j = n − 1) 2
⎛ ⎜ t1 = σB2 (t j = n − 2) + ∑ pm × ⎜ ∑ m = t j − 1+ 1 ⎜⎜ m = t j − 1+ 1 ⎝ t1
⎛ ⎞ ⎜ ⎟ mpm t1 ⎜ ⎟− p ∑ ⎜ m⎟ ⎝ m = t j − 1+ 1 ⎠
L
∑ m=1
⎞ ⎟ (mpm ) ⎟ ⎟⎟ ⎠ (8)
σB2 (t j = n ) 2
⎛ L ⎜ 2 = σB (t j = n − 1) + ∑ pm × ⎜ ∑ m = t j − 1+ 1 ⎜⎜ m = t j − 1+ 1 ⎝ L
ωj =
L
∑ m=1
⎞ ⎟ (mpm ) ⎟ ⎟⎟ ⎠ (9)
It can be known from Eqs. (7)–(9) that an n -scale threshold problem is reduced to a function optimization problem, and the threshold t j (particle) is found to maximize the objective function σB2 between different classes. The corresponding fitness function is as follows:
Fig. 4. SiC coated C/C composite material with flat circular blind hole defects.
⎧ ⎪ ⎪ ⎪
⎛ ⎞ ⎜ ⎟ mpm t1 ⎜ ⎟− ⎜ ∑ pm ⎟ ⎝ m = t j − 1+ 1 ⎠
max
tj
1 < t 1 <⋯< tn − 1 < L
σB2 (t j )
(10)
∑ pm , j = 1 m=1 tj
∑ pm , 1 < j < n ⎨ m = t j − 1+ 1 ⎪ L ⎪ ∑ pm , j = n ⎪ m=t +1 j−1 ⎩
3. Algorithm flow analysis 3.1. PSO algorithm (5)
Set in a N -dimension of the target search space, the position of the particle in the N -dimension space is represented as a vector Xi = (x1, x2, ⋯, xN ) , and the flight speed is expressed as Vi = (v1, v2, ⋯vn ) . Each particle has a fitness value determined by the objective function, and a velocity determines the direction and distance 3
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
Table 1 Defect size of the SiC coating – C/C composite matrix. Column Row
1
2
3
4
5
6
7
1
DA = 1.0 DH = 0.5
DA = 4.0 DH = 2.5
DA = 2.0 DH = 2.0
DA = 3.0 DH = 2.5
DA = 3.0 DH = 1.5
DA = 2.0 DH = 2.5
DA = 4.0 DH = 1.0
2
DA = 1.0 DH = 1.0
DA = 4.0 DH = 3.0
DA = 2.0 DH = 2.0
DA = 3.0 DH = 3.0
DA = 3.0 DH = 1.5
DA = 2.0 DH = 3.0
DA = 4.0 DH = 1.0
3
DA = 1.0 DH = 1.5
DA = 4.0 DH = 4.0
DA = 2.0 DH = 2.0
DA = 3.0 DH = 4.0
DA = 3.0 DH = 1.5
DA = 2.0 DH = 4.0
DA = 4.0 DH = 1.0
4
DA = 1.0 DH = 2.0
DA = 4.0 DH = 5.0
DA = 2.0 DH = 2.0
DA = 3.0 DH = 5.0
DA = 3.0 DH = 1.5
DA = 2.0 DH = 5.0
DA = 4.0 DH = 1.0
Note: DA-defect diameter (mm); DH-defect depth (mm).
replicated in large numbers, and the poor population will be gradually eliminated. In order to analyze the state of each population, it is necessary to calculate the fitness value of all particles of the population, and update the individual optimal value and the corresponding population optimal value. If the particle finds a new and better state [17,20], the particle will be copied in large numbers, and if the population cannot find the next better state, the particle will continue to be eliminated. The global optimization of the problem is realized by dynamically changing the current population size and the current search domain of the individual. The speed iteration formula of the DPSO algorithm is shown in Eq. (13) [21].
Table 2 PSO, DPSO, FODPSO parameter initialization values. Parameter
PSO
DPSO
FODPSO
Number of iterations Maximum number of iterations that are not improved Maximum number of iterations that are not improved Particle social behavior weight Current number of groups vmax vmin x max x min Number of particles Minimum number of particles Maximum number of particles Number of population Minimum population Maximum population Fractional coefficient Compression factor
150 – 0.8 0.8 150 5 −5 255 0 – – – 0.8 – – – 1.2
150 10 0.8 0.8 30 1.5 −1.5 255 0 4 2 6 30 10 50 – 1.2
150 10 0.8 0.8 30 1.5 −1.5 255 0 4 2 6 30 10 50 0.6 –
vi = αvi − 1 + ri ci (gbest − x i − 1) + r2 c2 (zbest − x i − 1 ) ⎫ x i = x i − 1 + 0.5vi ⎬ ⎭
The DPSO algorithm is the basic algorithm of the fractional Darwin particle swarm optimization algorithm FODPSO. The value of the algorithm coefficients can be understood by means of the binomial expansion coefficient [21], as shown in Eq. (14).
(1 + x )α
in which they fly. The particles follow the current optimal particle search in the solution space and know the best location and the current location that I have found so far. This can be seen as the particle's own flight experience. Each particle also knows the best location found by all particles in the entire population so far. It can be seen as the experience of particle companion. Particles determine the next move through their own experience and the best experience of their peers. For the k iteration, each particle in the PSO changes according to the following formula [19].
α (α − 1) 2 α (α − 1)(α − 2) x + 2! 3! α (α − 1)(α − 2)(α − 3) 4 3 x + x +⋯ 4!
= 1 + αx 1 +
(14)
When α = 1, the formula (14) can be simplified to the formula (15).
(1 + x )α = 1 + αx 1
(15)
According to the analogy of Eq. (13), the iteration speed formula is as shown in Eq. (16).
vidk+ 1 = w × vidk + c1 rand () × (pid − x idk ) + c2 × rand () × (pgd − x idk )
(16)
vi = 1 + αvi − 1 (11)
x idk+ 1 = x idk + vidk+ 1
(13)
Comparing the basic PSO algorithm iteration formula, the DPSO particle update speed formula can be derived as
(12)
vi = ri ci (gbest − x i − 1) + r2 c2 (zbest − x i − 1) + αvi − 1
In Eqs. (11) and (12), i = 1, 2, ⋯, M . M is the total number of particles in the group; vidk is the d dimension component of the flight velocity vector of the k iteration of the particle i ; x idk is the k The d component of the position vector of the iterative particle i ; pgd is the d dimension component of the best position of the particle i ; pgd is the d dimension component of the best position of the group; c1, c2 are weighting factors; rand() is a random function that produces a random number of [0,1]; w is an inertia weight function.
(17)
where α is a weight coefficient. 3.3. FODPSO algorithm FODPSO is an extended DPSO algorithm in which the fractional order α can be used to control the convergence speed of the algorithm. Fractional calculus (FC) has been applied to different fields such as mechanical engineering, computational mathematics, and fluid mechanics. For the sequence x (t ) , the fractional calculus based on Grunwald - Letnikov is defined as follows [22]:
3.2. DPSO algorithm The Darwin particle swarm DPSO algorithm is a PSO algorithm that uses a fusion natural selection operator. In the DPSO algorithm, when the particle optimization falls into the local optimum, the search domain is abandoned and replaced by other search domains. Based on this method, in each iteration cycle, a good population will be selected and
D α [x (t )] =
1 Tα
r
∑ k=0
(−1) k Γ(α + 1) x (t − kT ) Γ(k + 1)Γ(α − k + 1)
(18)
where α ∈ Const is the fractional order, Γ is the gamma function, T is 4
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
(a1) PSO
(b1) PSO
(a2) DPSO
(b2) DPSO
(a3) FODPSO
(b3) FODPSO (b) Image segmentation result when 4-level
(a) Image segmentation result when 3-level
Fig. 5. Image segmentation results.
the sampling period, and r is the truncation order. Grunwald - Letnikov pointed out that FODPSO has an important property: an integer-order calculus represents a finite series, and a fractional-order calculus represents an infinite series. Therefore, integer-order calculus is a local operator, and fractional-order calculus is a representation of all past states, and as time goes on, the influence of past states is gradually weakened [16–18,23]. For FODPSO, update the equation as follows [17]:
vi (t + 1) = c1 r1 (t )(pi (t ) − x i (t )) + c2 r2 (t )(pg (t ) − x i (t )) +
1 αvi (t 2
1
− 1) + 6 α (1 − α ) vi (t − 2) +
1 α (1 24
− α )(2 − α ) vi (t − 3) (19)
5
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
(c1) PSO
(d1) PSO
(c2) DPSO
(d2) DPSO
(d3) FODPSO (d) Image segmentation result when 6-level
(c3) FODPSO (c) Image segmentation result when 5-level Fig. 5. (continued)
binomial of Eq. (20), so there are many variations to the improvement of the fractional-order intelligent algorithm. Eq. (19) clearly defines the essence of the fractional order Darwin PSO algorithm, and the finite process is analogous to the general PSO algorithm. When α = 1, that is, the system has no memory of t − 2 and t − 3 in the past, FODPSO becomes DPSO, so the value of α greatly affects the
(1 + x )α a (a − 1) 2 a (a − 1)(a − 2) = 1 + αx 1 + x + 2! 3! a (a − 1)(a − 2)(a − 3) 4 3 x + x +⋯ 4!
(20)
The coefficients in Eq. (19) can be understood by the expansion of the 6
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
(e1) PSO
(f1) PSO
(e2) DPSO
(f2) DPSO
(e3) FODPSO (e) Image segmentation result when 21-level
(f3) FODPSO (f) Image segmentation result when 22-level
Fig. 5. (continued)
inertial particles. If the value of α is small, the particles ignore the dynamic characteristics of the system and are easily trapped in local optimum. If the value of α is relatively large, the particle will exhibit α variety of behaviors, so that the system can find the global optimal value, so in actual operation, the value of α should be increased. However, if the value of α is too large, the system execution time will be lengthened. Based on the initial experiments, when the fractional order α is within [0.5], the algorithm can achieve faster convergence speed, measured by all aspects of the performance, α = 0.6 system is optimal. It can be seen from Eq. (3) that for an image, the key is to find n − 1
Table 3 The number of defects recognized by PCA, PSO, DPSO and FODPSO algorithms at n = 22. Algorithm
PCA
PSO
DPSO
FODPSO
Number of defects
10
11
11
13
7
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
the pulse power supply, the internal capacitor discharges, and the flash lamp is triggered to emit the pulse thermal excitation at the moment of discharging. At the same time, the infrared thermal imager begins to heat the surface of the component under examination. The degree change is recorded and the heat wave signal is transmitted to the computer. The operator can use image processing and analysis software to process the image sequence. Under room temperature, standard atmospheric pressure and natural convection conditions, pulse energy 2280 J, sampling frequency 100 Hz and analysis time 1 s are set to test, and an image sequence is obtained. Then PCA algorithm is used to process the image sequence. Fig. 3 is the image processed by PCA algorithm. We have shown that PCA in combination with subtracting background sampling can be used to systematically model and subtract the thermal background from high–contrast imaging data-sets on a frameby-frame basis. It is an improvement over the conventional mean background subtraction approach because it can remove any fast changing in-homogeneous residual background. The PCA is used to find an orthogonal set of basis images (principal components) to model the residual background that is left after the mean background subtraction. The advantage of PCA is that it creates the basis set automatically and arranges the basis images according to their contribution to the representation of the background. Therefore, one can easily identify those principal components that should be chosen to model the residual background structure most effectively. The PCA is essentially a singular value decomposition (SVD) of the residual background images. The most important principal components are the orthonormal basis vectors that belong to the highest singular values [24]. Fig. 4 shows the SiC coated C/C composite material with flat circular blind hole defects. The defect size of the SiC coating – C/C composite matrix are listed in Table 1. Also, we should note that different methods listed in Table 2 have all sorts of variations in their experimental setups [25], n represents the image threshold scale. Both the computation and the algorithm were run in a PC [Intel(R) Core(TM) i3-6100 CPU, 3.70 GHz, RAM 4.00 GB, 64bit operating system], and the processing was conducted using MATLAB computer program. The results of segmentation and the simulation results of each step of the proposed algorithm are shown in Fig. 5 and Table 3 [26]. In the parameter setting, the number of thresholds can be freely set, and the image segmentation scale discussed is 3 ⩽ n ⩽ 22 , and the segmentation scale n is to divide the image into n parts. For the image of a thermal image collected in the test piece, the image threshold is optimized by PSO, DPSO and FODPSO algorithms. The image segmentation scale n is selected 3, 4, 5, 6, 21 and 22 respectively. As shown in Fig. 5, For the FODPSO algorithm, when the number of thresholds is 22, FODPSO algorithm can detect 13 defects, DPSO algorithm can detect 11 defects and PSO can detect 11 defects. Comparisons show that FODPSO has the best effect in identifying defects. In order to compare the defect signal-to-noise ratio (SNR) of different feature images, the feature images are normalized. The defect corresponding surface area is selected as the defect area, and the defectfree reference area is the shadow area shown in Fig. 6. In order to compare the advantages and disadvantages of several algorithms, the infrared thermal wave nondestructive testing results of matrix defects of SiC coated C/C composites are processed. The defect signal-to-noise ratio of image is used to measure the defect. The higher the defect signal-to-noise ratio of feature image is, the stronger the ability of suppressing noise and detecting is, the easier the defect can be judged and distinguished.
Fig. 6. Reference regions.
Fig. 7. Effect of characteristic information extraction algorithms on defect SNR.
Table 4 Effect of different algorithms on signal-to-noise ratio of defects. Defects
PCA
PSO
DPSO
FODPSO
#S1 #S2 #S3
0.7382 19.5285 7.3372
1.1000 21.8429 6.2286
0.6704 6.7787 1.5343
1.7571 24.7000 10.0262
different threshold scales t j to maximize the variance function σB2 between different classes. t j is equivalent to the particle in the FODPSO algorithm, and σB2 is the evaluation function of the particle, that is, the fitness function [17]. The FODPSO algorithm flow chart is shown in Fig. 1. 4. Pulsed infrared image segmentation Pulse infrared thermal wave nondestructive testing system is generally composed of thermal excitation system, thermal imaging system and infrared image sequence processing and analysis system. Fig. 2 shows the diagram of optical pulsed infrared thermal wave nondestructive testing system. The thermal excitation system mainly includes pulse power supply, flash lamp and mask, synchronous trigger and clamping device of components. The use of synchronous trigger can ensure that the pulse excitation keeps synchronization with the record of infrared thermal imager, which is conducive to the subsequent processing and analysis of infrared image sequence. The trigger signal from the recording software is collected by the infrared thermal imager. The trigger signal is synthesized into the pulse power supply synchronously through the synchronous trigger. After the trigger signal is received by
SNR =
PD − PN σN
(21)
where PD is the characteristic means of defect corresponding regions, PN is the characteristic means of defect-free corresponding regions, σN is the standard deviation of eigenvalues for defect-free regions, SNR is the signal to noise ratio of defects in characteristic images. 8
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
The result of this in shown in Fig. 7 and in Table 4. The rectangular defects represented by S1, S2 and S3 is shown in Fig. 6, respectively. For the three defect locations selected, the signal-to-noise ratio calculated by FODPSO algorithm is the largest, and the effect of defect detection is the best. With S N ≈ 24.7 , even in the best results.
[5]
[6] [7]
5. Conclusion
[8]
In this paper, the real infrared image captured by infrared camera is taken as the research object. PSO, DPSO and FODPSO are used to segment the image. The results show that the FODPSO algorithm is superior to the other two algorithms, and the number of defects detected is the largest and the signal-to-noise ratio can be improved. Therefore, FODPSO plays an important role in infrared image segmentation and defect edge recognition. With the rapid development of mathematics and computer science, more advanced signal processing algorithms will emerge, and defect feature extraction can be realized more effectively.
[9]
[10]
[11]
[12]
Declaration of Competing Interest [13]
There is no conflict of interest. [14]
Acknowledgements
[15]
This project is supported by National Natural Science Foundation of China (Grant No. 51775175), Heilongjiang Province Natural Science Fund (Grant No. E2018050), Heilongjiang Science and Technology Plan Provincial Hospital Science and Technology Cooperation Project (Grant No. YS18A18), and Harbin Science and Technology Bureau Fund (Grant No. 2017RAXXJ015).
[16] [17] [18] [19]
Appendix A. Supplementary material [20]
Supplementary data to this article can be found online at https:// doi.org/10.1016/j.infrared.2019.103051.
[21] [22]
References [23] [1] H. Fernandes, H. Zhang, C. Ibarra-Castanedo, et al., Fiber orientation assessment on randomly-oriented strand composites by means of infrared thermography [J], Compos. Sci. Technol. 121 (2015) 25–33. [2] B. Yousefi, S. Sfarra, F. Sarasini, et al., IRNDT inspection via sparse principal component thermography [C], in: IEEE Canadian Conference on Electrical & Computer Engineering (CCECE), 2018. [3] H. Zhang, S. Sfarra, K. Saluja, et al., Non-destructive investigation of paintings on canvas by continuous wave terahertz imaging and flash thermography [J], J. Nondestr. Eval. 36 (2) (2017) 34. [4] Q. Tang, J. Liu, Y. Wang, et al., Inspection on SiC coated carbon–carbon composite
[24] [25] [26]
9
with subsurface defects using pulsed thermography [J], Infrared Phys. Technol. 60 (60) (2013) 183–189. Q. Tang, J. Dai, J. Liu, et al., Quantitative detection of defects based on Markov – PCA – BP algorithm using pulsed infrared thermography technology [J], Infrared Phys. Technol. 77 (2016) 144–148. M. Barus, H. Welemane, V. Nassiet, et al., NDT-based design of joint material for the detection of bonding defects by infrared thermography [J], Ndt & E Int. (2018) 93. Rubén Usamentiaga, P. Venegas, J. Guerediaga, et al., Infrared thermography for temperature measurement and non-destructive testing [J], Sensors 14 (7) (2014) 12305–12348. N. Rajic, Principal component thermography for flaw contrast enhancement and flaw depth characterisation in composite structures [J], Compos. Struct. 58 (4) (2002) 521–528. B. Yousefi, S. Sfarra, C. Ibarra Castanedo, et al., Comparative analysis on thermal non-destructive testing imagery applying Candid Covariance-Free Incremental Principal Component Thermography (CCIPCT) [J], Infrared Phys. Technol. 85 (2017) 163–169. H. Gao, W. Xu, J. Sun, et al., Multilevel thresholding for image segmentation through an improved quantum-behaved particle swarm algorithm [J], IEEE Trans. Instrum. Meas. 59 (4) (2010) 934–946. Y. Li, L. Jiao, R. Shang, et al., Dynamic-context cooperative quantum-behaved particle swarm optimization based on multilevel thresholding applied to medical image segmentation [J], Inf. Sci. 294 (2015) 408–422. N. Liu, Y. Jiang, S. Li, Application of PSO algorithm with dynamic inertia weight in medical image thresholding segmentation [C], in: International Conference on EHealth Networking, Applications and Services. IEEE, 2017, pp. 1–4. X. Xie, J. Wang, F. Hu, An improved floc image segmentation algorithm based on particle swarm optimization and entropic [J], Int. J. Comput. Sci. Eng. 15 (1/2) (2017) 49. E. Küçükkülahli, P. Erdoğmuş, K. Polat, Brain MRI segmentation based on different clustering algorithms [J], Int. J. Comput. Appl. 155 (3) (2017) 0975–8887. J. Tillett, T.M. Rao, F. Sahin, et al., Darwinian particle swarm optimization[C], in: Indian International Conference on Artificial Intelligence, Pune, India, December. DBLP, 2005, pp. 1474–1487. J.A. Pires, P.B. Moura, A.M. Oliveir, et al., Particle swarm optimization with fractional-order velocity [J], Nonlinear Dyn. 61 (1–2) (2010) 295–301. S. Yu, Z. Cao, Image segmentation based on fractional-order Darwinian particle swarm optimization [J], Comput. Eng. Sci. 38 (09) (2016) 1836–1842. S.C. Lingam, S.B. Belhaouari, Detection of kidney fault using threshold segmentation method [J], Int. J. Pharma Bio Sci. 8 (3) (2017). L. Yu, Image Segmentation Based on Particle Swarm Optimization and Ant Colony Algorism Based on GPU-Accelerated [D], Dalian University of Technology, Da Lian, 2007. J.G. Kang, A study on an arrangement of passive sonars by using DPSO algorithm [J], J. Korea Soc. Simulat. 26 (1) (2017) 39–46. S. Yu, DSP for image segmentation based on FODPSO algorithm [D], Southwest JiaoTong University, ChengDu, 2016. P. Ghamisi, M.S. Couceiro, J.A. Benediktsson, et al., An efficient method for segmentation of images based on fractional calculus and natural selection [J], Expert Syst. Appl. 39 (16) (2012) 12407–12417. P. Ghamisi, M.S. Couceiro, J.A. Benediktsson, A novel feature selection approach based on FODPSO and SVM [J], IEEE Trans. Geosci. Remote Sens. 53 (5) (2015) 2935–2947. S. Hunziker, S.P. Quanz, A. Amera, A PCA-based approach for subtracting thermal background emission in high-contrast imaging data [J], A & A 29 (9) (2017) 1–12. B. Yousefi, K.L. Chu, Bio-inspired human action recognition using hybrid maxproduct Neuro-Fuzzy classifier and quantum-behaved PSO [J]. arxiv 2015. Y. Bardia, S. Stefano, I.C. Clemente, et al., Thermography data fusion and nonnegative matrix factorization for the evaluation of cultural heritage objects and buildings [J], J. Therm. Anal. Calorim. (2018).
Contents lists available at ScienceDirect
Infrared Physics & Technology journal homepage: www.elsevier.com/locate/infrared
Infrared image segmentation algorithm for defect detection based on FODPSO
T
⁎
Qingju Tang , Shuaishuai Gao, Yongjie Liu, Fengyun Yu School of Mechanical Engineering, Heilongjiang University of Science and Technology, Harbin 150022, PR China
A R T I C LE I N FO
A B S T R A C T
Keywords: Fractional order Fractional-order Darwin particle swarm algorithm Infrared image Edge detection Multi-scale segmentation Algorithm comparison SiC coating
To address the problems of high noise and fuzzy edges of acquired infrared images when detecting debonding defective components of SiC thermal barrier coating structure by pulsed infrared thermal waves, an image segmentation method based on fractional Darwin particle swarm optimization algorithm (FODPSO) is adopted to reconstruct infrared image sequences. This algorithm adopts fractional calculus to control the convergence of the system, and can perform n − 1 threshold optimization calculation on n − 1 level images. Experimental results show that FODPSO algorithm effectively overcomes the shortcomings of traditional algorithm such as easy to enter local optimal and slow convergence speed, and can realize the edge identification of SiC coating defects. The processed image improves the contrast and signal-to-noise ratio.
1. Introduction Infrared thermography (IT) is a safe non-destructive testing (NDT) technique that has a fast inspection rate and is generally contact-less. It is used for diagnostics and monitoring in several fields such as electrical components, thermal comfort, buildings, artworks, medical, security and composite materials [1], and in most of these applications wellknown infrared approaches have been utilized for thermal image enhancement, thermal image segmentation, and particularly defect segmentation in IRNDT [2]. In the process of detecting the structural defects of the pulsed infrared heat wave, the surface of the component is heated by the instantaneous heat pulse emitted by the high-energy flash lamp. It can be combined with imaging in order to discriminate between materials and/or attend to “determine a period of time” of execution for an object [3], and the image changes of the surface temperature field is recorded by the infrared camera. The thermal image sequence is analyzed and processed to observe internal defects or structural discontinuities in the component. The detection of defects in the original thermal image collected directly from the infrared camera is the main method of IR NDT – infrared thermographic nondestructive testing [4–6]. The difference between a visible image and an infrared image is that the visible image is a representation of the reflected light on the scene, whereas in the infrared image, the scene is the source and can be observed by an infrared camera without light. Images acquired using infrared cameras are converted into visible images by assigning a color to each infrared energy level [7]. The technique is based on
⁎
remote infrared detection of surface temperature anomalies that evolve in response to the application of a thermal perturbation to the structure. Relative to radiography and ultrasound, the most commonly applied nondestructive inspection techniques for composite materials, thermography offers significant gains in both ease and speed of inspection [8]. However, because the surface temperature caused by defects or structural anomalies with smaller diameters is very small compared to the background temperature, that is, the original thermal image is a typical low-contrast image, its ability to identify local abnormal changes inside the specimen is very limited. Segmentation is essential to identify the defects after applying the prepossessing techniques. This section briefly reviews the methods employed for segmentation [9]. Image segmentation is the main technique in image processing. The method is to separate the target from the background and provide a basis for subsequent classification, recognition and retrieval. Image segmentation applies one or more operations to divide the image into regions with similar characteristics. Image segmentation methods mainly include threshold methods and regional methods. The threshold method uses the gray frequency information to segment the distribution information, and the region method uses the local spatial information to perform segmentation. The image threshold method is the most used and simplest method. Optimization plays an important role in computer science, artificial intelligence, operational research, and other related fields. It is the process of trying to find the best possible solution to an optimization problem within a reasonable time limit [10]. The particle swarm optimization (PSO) algorithm is a stochastic global optimization
Corresponding author. E-mail address: [email protected] (Q. Tang).
https://doi.org/10.1016/j.infrared.2019.103051 Received 25 November 2018; Received in revised form 13 August 2019; Accepted 23 September 2019 Available online 23 September 2019 1350-4495/ © 2019 Elsevier B.V. All rights reserved.
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
calculation method based on swarm intelligence. The algorithm has the advantages of fast convergence and easy implementation. A particularly important capability is image segmentation. Segmentation is the process of partitioning an image into a set of non-intersecting regions, such that each region is homogeneous, but the union of no two adjacent regions is homogeneous. This is a fundamental problem in Computer Vision, and a number of methods have been proposed for solving it [11]. However, it also shows some problems to be improved in practical applications, that is, the algorithm tends to converge quickly in the early stage of search, but in the later stage, the particle group will tend to be the same (Loss of diversity), the convergence speed is obviously slower, the search accuracy is reduced, and the algorithm is easy to fall into local optimum. Therefore, many scholars are working to improve the performance of the PSO algorithm [12–14]. When performing the image segmentation, how to determine the number of categories of image segmentation that has always been a hot and difficult point of research. Tillett et al. [15] proposed a Darwinian particle swarm optimization (DPSO) algorithm that uses natural selection evolution to dynamically divide a population into subgroups. Each subgroup is searched independently to improve particle diversity and enhance the global optimization ability of the algorithm. Pires et al. [16] proposed FODPSO is an extensible DPSO algorithm, in which fractional order α is used to control the convergence speed of the algorithm. It has an important property: on the one hand, an integer order calculus represents a finite series. On the other hand, a fractional order calculus represents an infinite term series. Therefore, integer-order calculus is a local operator, and fractional-order calculus is a representation of all the states in the past. With the increase of time, the influence of the past states gradually weakens. Because of the inherent nature of fractional-order calculus, it can describe the irreversibility and chaos well, making the particle trajectory more suitable for the global optimal situation. The fitness values of initial individuals are calculated, the particle velocity updates and fitness values are calculated, and compared with the previous generation optimal fitness values, the optimal fitness of the population is updated until the number of iterations is reached. The FODPSO algorithm has better computational accuracy and convergence speed than PSO and DPSO (Darwinian Particle Swarm Optimization). The convergence performance of FODPSO algorithm depends on the fractional order directly. When α increasing, the convergence speed becomes slower, and when it decreases, the population falls into the local maximum. The probability of superiority becomes higher.
Fig. 1. FODPSO algorithm flow chart.
threshold scales t j , j = 1, 2, 3⋯,n − 1, the image after segmentation is as follows:
⎧ ⎪ ⎪ F (x , y ) =
2. n-Scale threshold segmentation model The robustness of n-scale threshold optimization is a difficult point for image segmentation. Common image gray values are 0–255. Set a maximum gray value L = 256, then for a gray image, the gray threshold can be taken as {0, 1, 2, …, L − 1}, correspondingly, pm , the probability of each gray value, could be expressed as [17,18].
pm =
hm , N
m−1
+ t2), t1 < f (x , y ) ⩽ t2
⋮ ⎨1 ⎪ 2 (tn − 2 + tn − 1), tn − 2 < f (x , y ) ⩽ tn − 1 ⎪ L, f (x , y ) > tn − 1 ⎩
(3)
where the function f (x , y ) represents the gray value corresponding to the image coordinate (x , y ) , x corresponds to the width W coordinate, y corresponds to the height H coordinate; F (x , y ) is the divided image gray value, and the image is divided into D1, D2 , ⋯Dn types. It can be known from Eq. (3) that the image can be segmented by finding n − 1 different threshold scales t j , and how to find the threshold evaluation function (adaptive function) becomes the next problem to be solved [12,13,17]. The simplest and fastest threshold optimization method is to directly find the gray value corresponding to the largest variance between different classes, which is generally defined as follows:
N
∑ pm
0, f (x , y ) ⩽ t1 1 (t 2 1
=1 (1)
where m is the specific image gray value, 0 ⩽ m ⩽ L − 1; N is the number of pixels in the image, N = H × W ; hm is the total number of gray values of the image, hm can be directly obtained from the statistical histogram; then the average value of the gray is:
n
σB2 =
L
μT =
∑ mpm m=1
∑ ωj (μj − σT )2 j=1
(2)
(4)
where j represents a concrete class, ωj represents the probability of the j class, μj is the mean of the j class, and ωj and μj of the specific D1, D2 , ⋯Dn class are expressed as follows [17]:
According to the calculated threshold μT , the image gray value is greater than or equal to μT divided into one category, and less than μT is divided into other category, which becomes a common image binarization process. Here, the two-dimensional threshold scale problem is extended to the n -dimensional threshold scale, that is n − 1 different 2
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
Flash lamp
Infrared Thermal Imager
Thermal wave signal Computer
Trigger signal
Trigger signal
Specimen Synchronous trigger Mask
Pulse power supply
Fig. 2. Diagram of optical pulsed infrared thermal wave nondestructive testing system.
μj =
⎧ ⎪ ⎪ ⎪
tj
∑ m=1 tj
mpm ωj
,j=1
mp
m ,1
∑
(6)
From Eqs. (4)–(6), Eqs. (7)–(9) are available: 2
⎛ t ⎜ 1 2 2 σB (t j = 1) = ωj (μj − μT ) = ∑ pm × ∑ ⎜ m=1 m=1 ⎜ ⎝ t1
⎛ ⎞ ⎜ mpm ⎟ − ⎜ t1 ⎟ ⎜ ∑ pm ⎟ ⎝ m=1 ⎠
L
∑ m=1
⎞ ⎟ (mpm ) ⎟ ⎟ ⎠ (7)
Fig. 3. Input infrared image processed by PCA.
σB2 (t j = n − 1) 2
⎛ ⎜ t1 = σB2 (t j = n − 2) + ∑ pm × ⎜ ∑ m = t j − 1+ 1 ⎜⎜ m = t j − 1+ 1 ⎝ t1
⎛ ⎞ ⎜ ⎟ mpm t1 ⎜ ⎟− p ∑ ⎜ m⎟ ⎝ m = t j − 1+ 1 ⎠
L
∑ m=1
⎞ ⎟ (mpm ) ⎟ ⎟⎟ ⎠ (8)
σB2 (t j = n ) 2
⎛ L ⎜ 2 = σB (t j = n − 1) + ∑ pm × ⎜ ∑ m = t j − 1+ 1 ⎜⎜ m = t j − 1+ 1 ⎝ L
ωj =
L
∑ m=1
⎞ ⎟ (mpm ) ⎟ ⎟⎟ ⎠ (9)
It can be known from Eqs. (7)–(9) that an n -scale threshold problem is reduced to a function optimization problem, and the threshold t j (particle) is found to maximize the objective function σB2 between different classes. The corresponding fitness function is as follows:
Fig. 4. SiC coated C/C composite material with flat circular blind hole defects.
⎧ ⎪ ⎪ ⎪
⎛ ⎞ ⎜ ⎟ mpm t1 ⎜ ⎟− ⎜ ∑ pm ⎟ ⎝ m = t j − 1+ 1 ⎠
max
tj
1 < t 1 <⋯< tn − 1 < L
σB2 (t j )
(10)
∑ pm , j = 1 m=1 tj
∑ pm , 1 < j < n ⎨ m = t j − 1+ 1 ⎪ L ⎪ ∑ pm , j = n ⎪ m=t +1 j−1 ⎩
3. Algorithm flow analysis 3.1. PSO algorithm (5)
Set in a N -dimension of the target search space, the position of the particle in the N -dimension space is represented as a vector Xi = (x1, x2, ⋯, xN ) , and the flight speed is expressed as Vi = (v1, v2, ⋯vn ) . Each particle has a fitness value determined by the objective function, and a velocity determines the direction and distance 3
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
Table 1 Defect size of the SiC coating – C/C composite matrix. Column Row
1
2
3
4
5
6
7
1
DA = 1.0 DH = 0.5
DA = 4.0 DH = 2.5
DA = 2.0 DH = 2.0
DA = 3.0 DH = 2.5
DA = 3.0 DH = 1.5
DA = 2.0 DH = 2.5
DA = 4.0 DH = 1.0
2
DA = 1.0 DH = 1.0
DA = 4.0 DH = 3.0
DA = 2.0 DH = 2.0
DA = 3.0 DH = 3.0
DA = 3.0 DH = 1.5
DA = 2.0 DH = 3.0
DA = 4.0 DH = 1.0
3
DA = 1.0 DH = 1.5
DA = 4.0 DH = 4.0
DA = 2.0 DH = 2.0
DA = 3.0 DH = 4.0
DA = 3.0 DH = 1.5
DA = 2.0 DH = 4.0
DA = 4.0 DH = 1.0
4
DA = 1.0 DH = 2.0
DA = 4.0 DH = 5.0
DA = 2.0 DH = 2.0
DA = 3.0 DH = 5.0
DA = 3.0 DH = 1.5
DA = 2.0 DH = 5.0
DA = 4.0 DH = 1.0
Note: DA-defect diameter (mm); DH-defect depth (mm).
replicated in large numbers, and the poor population will be gradually eliminated. In order to analyze the state of each population, it is necessary to calculate the fitness value of all particles of the population, and update the individual optimal value and the corresponding population optimal value. If the particle finds a new and better state [17,20], the particle will be copied in large numbers, and if the population cannot find the next better state, the particle will continue to be eliminated. The global optimization of the problem is realized by dynamically changing the current population size and the current search domain of the individual. The speed iteration formula of the DPSO algorithm is shown in Eq. (13) [21].
Table 2 PSO, DPSO, FODPSO parameter initialization values. Parameter
PSO
DPSO
FODPSO
Number of iterations Maximum number of iterations that are not improved Maximum number of iterations that are not improved Particle social behavior weight Current number of groups vmax vmin x max x min Number of particles Minimum number of particles Maximum number of particles Number of population Minimum population Maximum population Fractional coefficient Compression factor
150 – 0.8 0.8 150 5 −5 255 0 – – – 0.8 – – – 1.2
150 10 0.8 0.8 30 1.5 −1.5 255 0 4 2 6 30 10 50 – 1.2
150 10 0.8 0.8 30 1.5 −1.5 255 0 4 2 6 30 10 50 0.6 –
vi = αvi − 1 + ri ci (gbest − x i − 1) + r2 c2 (zbest − x i − 1 ) ⎫ x i = x i − 1 + 0.5vi ⎬ ⎭
The DPSO algorithm is the basic algorithm of the fractional Darwin particle swarm optimization algorithm FODPSO. The value of the algorithm coefficients can be understood by means of the binomial expansion coefficient [21], as shown in Eq. (14).
(1 + x )α
in which they fly. The particles follow the current optimal particle search in the solution space and know the best location and the current location that I have found so far. This can be seen as the particle's own flight experience. Each particle also knows the best location found by all particles in the entire population so far. It can be seen as the experience of particle companion. Particles determine the next move through their own experience and the best experience of their peers. For the k iteration, each particle in the PSO changes according to the following formula [19].
α (α − 1) 2 α (α − 1)(α − 2) x + 2! 3! α (α − 1)(α − 2)(α − 3) 4 3 x + x +⋯ 4!
= 1 + αx 1 +
(14)
When α = 1, the formula (14) can be simplified to the formula (15).
(1 + x )α = 1 + αx 1
(15)
According to the analogy of Eq. (13), the iteration speed formula is as shown in Eq. (16).
vidk+ 1 = w × vidk + c1 rand () × (pid − x idk ) + c2 × rand () × (pgd − x idk )
(16)
vi = 1 + αvi − 1 (11)
x idk+ 1 = x idk + vidk+ 1
(13)
Comparing the basic PSO algorithm iteration formula, the DPSO particle update speed formula can be derived as
(12)
vi = ri ci (gbest − x i − 1) + r2 c2 (zbest − x i − 1) + αvi − 1
In Eqs. (11) and (12), i = 1, 2, ⋯, M . M is the total number of particles in the group; vidk is the d dimension component of the flight velocity vector of the k iteration of the particle i ; x idk is the k The d component of the position vector of the iterative particle i ; pgd is the d dimension component of the best position of the particle i ; pgd is the d dimension component of the best position of the group; c1, c2 are weighting factors; rand() is a random function that produces a random number of [0,1]; w is an inertia weight function.
(17)
where α is a weight coefficient. 3.3. FODPSO algorithm FODPSO is an extended DPSO algorithm in which the fractional order α can be used to control the convergence speed of the algorithm. Fractional calculus (FC) has been applied to different fields such as mechanical engineering, computational mathematics, and fluid mechanics. For the sequence x (t ) , the fractional calculus based on Grunwald - Letnikov is defined as follows [22]:
3.2. DPSO algorithm The Darwin particle swarm DPSO algorithm is a PSO algorithm that uses a fusion natural selection operator. In the DPSO algorithm, when the particle optimization falls into the local optimum, the search domain is abandoned and replaced by other search domains. Based on this method, in each iteration cycle, a good population will be selected and
D α [x (t )] =
1 Tα
r
∑ k=0
(−1) k Γ(α + 1) x (t − kT ) Γ(k + 1)Γ(α − k + 1)
(18)
where α ∈ Const is the fractional order, Γ is the gamma function, T is 4
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
(a1) PSO
(b1) PSO
(a2) DPSO
(b2) DPSO
(a3) FODPSO
(b3) FODPSO (b) Image segmentation result when 4-level
(a) Image segmentation result when 3-level
Fig. 5. Image segmentation results.
the sampling period, and r is the truncation order. Grunwald - Letnikov pointed out that FODPSO has an important property: an integer-order calculus represents a finite series, and a fractional-order calculus represents an infinite series. Therefore, integer-order calculus is a local operator, and fractional-order calculus is a representation of all past states, and as time goes on, the influence of past states is gradually weakened [16–18,23]. For FODPSO, update the equation as follows [17]:
vi (t + 1) = c1 r1 (t )(pi (t ) − x i (t )) + c2 r2 (t )(pg (t ) − x i (t )) +
1 αvi (t 2
1
− 1) + 6 α (1 − α ) vi (t − 2) +
1 α (1 24
− α )(2 − α ) vi (t − 3) (19)
5
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
(c1) PSO
(d1) PSO
(c2) DPSO
(d2) DPSO
(d3) FODPSO (d) Image segmentation result when 6-level
(c3) FODPSO (c) Image segmentation result when 5-level Fig. 5. (continued)
binomial of Eq. (20), so there are many variations to the improvement of the fractional-order intelligent algorithm. Eq. (19) clearly defines the essence of the fractional order Darwin PSO algorithm, and the finite process is analogous to the general PSO algorithm. When α = 1, that is, the system has no memory of t − 2 and t − 3 in the past, FODPSO becomes DPSO, so the value of α greatly affects the
(1 + x )α a (a − 1) 2 a (a − 1)(a − 2) = 1 + αx 1 + x + 2! 3! a (a − 1)(a − 2)(a − 3) 4 3 x + x +⋯ 4!
(20)
The coefficients in Eq. (19) can be understood by the expansion of the 6
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
(e1) PSO
(f1) PSO
(e2) DPSO
(f2) DPSO
(e3) FODPSO (e) Image segmentation result when 21-level
(f3) FODPSO (f) Image segmentation result when 22-level
Fig. 5. (continued)
inertial particles. If the value of α is small, the particles ignore the dynamic characteristics of the system and are easily trapped in local optimum. If the value of α is relatively large, the particle will exhibit α variety of behaviors, so that the system can find the global optimal value, so in actual operation, the value of α should be increased. However, if the value of α is too large, the system execution time will be lengthened. Based on the initial experiments, when the fractional order α is within [0.5], the algorithm can achieve faster convergence speed, measured by all aspects of the performance, α = 0.6 system is optimal. It can be seen from Eq. (3) that for an image, the key is to find n − 1
Table 3 The number of defects recognized by PCA, PSO, DPSO and FODPSO algorithms at n = 22. Algorithm
PCA
PSO
DPSO
FODPSO
Number of defects
10
11
11
13
7
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
the pulse power supply, the internal capacitor discharges, and the flash lamp is triggered to emit the pulse thermal excitation at the moment of discharging. At the same time, the infrared thermal imager begins to heat the surface of the component under examination. The degree change is recorded and the heat wave signal is transmitted to the computer. The operator can use image processing and analysis software to process the image sequence. Under room temperature, standard atmospheric pressure and natural convection conditions, pulse energy 2280 J, sampling frequency 100 Hz and analysis time 1 s are set to test, and an image sequence is obtained. Then PCA algorithm is used to process the image sequence. Fig. 3 is the image processed by PCA algorithm. We have shown that PCA in combination with subtracting background sampling can be used to systematically model and subtract the thermal background from high–contrast imaging data-sets on a frameby-frame basis. It is an improvement over the conventional mean background subtraction approach because it can remove any fast changing in-homogeneous residual background. The PCA is used to find an orthogonal set of basis images (principal components) to model the residual background that is left after the mean background subtraction. The advantage of PCA is that it creates the basis set automatically and arranges the basis images according to their contribution to the representation of the background. Therefore, one can easily identify those principal components that should be chosen to model the residual background structure most effectively. The PCA is essentially a singular value decomposition (SVD) of the residual background images. The most important principal components are the orthonormal basis vectors that belong to the highest singular values [24]. Fig. 4 shows the SiC coated C/C composite material with flat circular blind hole defects. The defect size of the SiC coating – C/C composite matrix are listed in Table 1. Also, we should note that different methods listed in Table 2 have all sorts of variations in their experimental setups [25], n represents the image threshold scale. Both the computation and the algorithm were run in a PC [Intel(R) Core(TM) i3-6100 CPU, 3.70 GHz, RAM 4.00 GB, 64bit operating system], and the processing was conducted using MATLAB computer program. The results of segmentation and the simulation results of each step of the proposed algorithm are shown in Fig. 5 and Table 3 [26]. In the parameter setting, the number of thresholds can be freely set, and the image segmentation scale discussed is 3 ⩽ n ⩽ 22 , and the segmentation scale n is to divide the image into n parts. For the image of a thermal image collected in the test piece, the image threshold is optimized by PSO, DPSO and FODPSO algorithms. The image segmentation scale n is selected 3, 4, 5, 6, 21 and 22 respectively. As shown in Fig. 5, For the FODPSO algorithm, when the number of thresholds is 22, FODPSO algorithm can detect 13 defects, DPSO algorithm can detect 11 defects and PSO can detect 11 defects. Comparisons show that FODPSO has the best effect in identifying defects. In order to compare the defect signal-to-noise ratio (SNR) of different feature images, the feature images are normalized. The defect corresponding surface area is selected as the defect area, and the defectfree reference area is the shadow area shown in Fig. 6. In order to compare the advantages and disadvantages of several algorithms, the infrared thermal wave nondestructive testing results of matrix defects of SiC coated C/C composites are processed. The defect signal-to-noise ratio of image is used to measure the defect. The higher the defect signal-to-noise ratio of feature image is, the stronger the ability of suppressing noise and detecting is, the easier the defect can be judged and distinguished.
Fig. 6. Reference regions.
Fig. 7. Effect of characteristic information extraction algorithms on defect SNR.
Table 4 Effect of different algorithms on signal-to-noise ratio of defects. Defects
PCA
PSO
DPSO
FODPSO
#S1 #S2 #S3
0.7382 19.5285 7.3372
1.1000 21.8429 6.2286
0.6704 6.7787 1.5343
1.7571 24.7000 10.0262
different threshold scales t j to maximize the variance function σB2 between different classes. t j is equivalent to the particle in the FODPSO algorithm, and σB2 is the evaluation function of the particle, that is, the fitness function [17]. The FODPSO algorithm flow chart is shown in Fig. 1. 4. Pulsed infrared image segmentation Pulse infrared thermal wave nondestructive testing system is generally composed of thermal excitation system, thermal imaging system and infrared image sequence processing and analysis system. Fig. 2 shows the diagram of optical pulsed infrared thermal wave nondestructive testing system. The thermal excitation system mainly includes pulse power supply, flash lamp and mask, synchronous trigger and clamping device of components. The use of synchronous trigger can ensure that the pulse excitation keeps synchronization with the record of infrared thermal imager, which is conducive to the subsequent processing and analysis of infrared image sequence. The trigger signal from the recording software is collected by the infrared thermal imager. The trigger signal is synthesized into the pulse power supply synchronously through the synchronous trigger. After the trigger signal is received by
SNR =
PD − PN σN
(21)
where PD is the characteristic means of defect corresponding regions, PN is the characteristic means of defect-free corresponding regions, σN is the standard deviation of eigenvalues for defect-free regions, SNR is the signal to noise ratio of defects in characteristic images. 8
Infrared Physics and Technology 102 (2019) 103051
Q. Tang, et al.
The result of this in shown in Fig. 7 and in Table 4. The rectangular defects represented by S1, S2 and S3 is shown in Fig. 6, respectively. For the three defect locations selected, the signal-to-noise ratio calculated by FODPSO algorithm is the largest, and the effect of defect detection is the best. With S N ≈ 24.7 , even in the best results.
[5]
[6] [7]
5. Conclusion
[8]
In this paper, the real infrared image captured by infrared camera is taken as the research object. PSO, DPSO and FODPSO are used to segment the image. The results show that the FODPSO algorithm is superior to the other two algorithms, and the number of defects detected is the largest and the signal-to-noise ratio can be improved. Therefore, FODPSO plays an important role in infrared image segmentation and defect edge recognition. With the rapid development of mathematics and computer science, more advanced signal processing algorithms will emerge, and defect feature extraction can be realized more effectively.
[9]
[10]
[11]
[12]
Declaration of Competing Interest [13]
There is no conflict of interest. [14]
Acknowledgements
[15]
This project is supported by National Natural Science Foundation of China (Grant No. 51775175), Heilongjiang Province Natural Science Fund (Grant No. E2018050), Heilongjiang Science and Technology Plan Provincial Hospital Science and Technology Cooperation Project (Grant No. YS18A18), and Harbin Science and Technology Bureau Fund (Grant No. 2017RAXXJ015).
[16] [17] [18] [19]
Appendix A. Supplementary material [20]
Supplementary data to this article can be found online at https:// doi.org/10.1016/j.infrared.2019.103051.
[21] [22]
References [23] [1] H. Fernandes, H. Zhang, C. Ibarra-Castanedo, et al., Fiber orientation assessment on randomly-oriented strand composites by means of infrared thermography [J], Compos. Sci. Technol. 121 (2015) 25–33. [2] B. Yousefi, S. Sfarra, F. Sarasini, et al., IRNDT inspection via sparse principal component thermography [C], in: IEEE Canadian Conference on Electrical & Computer Engineering (CCECE), 2018. [3] H. Zhang, S. Sfarra, K. Saluja, et al., Non-destructive investigation of paintings on canvas by continuous wave terahertz imaging and flash thermography [J], J. Nondestr. Eval. 36 (2) (2017) 34. [4] Q. Tang, J. Liu, Y. Wang, et al., Inspection on SiC coated carbon–carbon composite
[24] [25] [26]
9
with subsurface defects using pulsed thermography [J], Infrared Phys. Technol. 60 (60) (2013) 183–189. Q. Tang, J. Dai, J. Liu, et al., Quantitative detection of defects based on Markov – PCA – BP algorithm using pulsed infrared thermography technology [J], Infrared Phys. Technol. 77 (2016) 144–148. M. Barus, H. Welemane, V. Nassiet, et al., NDT-based design of joint material for the detection of bonding defects by infrared thermography [J], Ndt & E Int. (2018) 93. Rubén Usamentiaga, P. Venegas, J. Guerediaga, et al., Infrared thermography for temperature measurement and non-destructive testing [J], Sensors 14 (7) (2014) 12305–12348. N. Rajic, Principal component thermography for flaw contrast enhancement and flaw depth characterisation in composite structures [J], Compos. Struct. 58 (4) (2002) 521–528. B. Yousefi, S. Sfarra, C. Ibarra Castanedo, et al., Comparative analysis on thermal non-destructive testing imagery applying Candid Covariance-Free Incremental Principal Component Thermography (CCIPCT) [J], Infrared Phys. Technol. 85 (2017) 163–169. H. Gao, W. Xu, J. Sun, et al., Multilevel thresholding for image segmentation through an improved quantum-behaved particle swarm algorithm [J], IEEE Trans. Instrum. Meas. 59 (4) (2010) 934–946. Y. Li, L. Jiao, R. Shang, et al., Dynamic-context cooperative quantum-behaved particle swarm optimization based on multilevel thresholding applied to medical image segmentation [J], Inf. Sci. 294 (2015) 408–422. N. Liu, Y. Jiang, S. Li, Application of PSO algorithm with dynamic inertia weight in medical image thresholding segmentation [C], in: International Conference on EHealth Networking, Applications and Services. IEEE, 2017, pp. 1–4. X. Xie, J. Wang, F. Hu, An improved floc image segmentation algorithm based on particle swarm optimization and entropic [J], Int. J. Comput. Sci. Eng. 15 (1/2) (2017) 49. E. Küçükkülahli, P. Erdoğmuş, K. Polat, Brain MRI segmentation based on different clustering algorithms [J], Int. J. Comput. Appl. 155 (3) (2017) 0975–8887. J. Tillett, T.M. Rao, F. Sahin, et al., Darwinian particle swarm optimization[C], in: Indian International Conference on Artificial Intelligence, Pune, India, December. DBLP, 2005, pp. 1474–1487. J.A. Pires, P.B. Moura, A.M. Oliveir, et al., Particle swarm optimization with fractional-order velocity [J], Nonlinear Dyn. 61 (1–2) (2010) 295–301. S. Yu, Z. Cao, Image segmentation based on fractional-order Darwinian particle swarm optimization [J], Comput. Eng. Sci. 38 (09) (2016) 1836–1842. S.C. Lingam, S.B. Belhaouari, Detection of kidney fault using threshold segmentation method [J], Int. J. Pharma Bio Sci. 8 (3) (2017). L. Yu, Image Segmentation Based on Particle Swarm Optimization and Ant Colony Algorism Based on GPU-Accelerated [D], Dalian University of Technology, Da Lian, 2007. J.G. Kang, A study on an arrangement of passive sonars by using DPSO algorithm [J], J. Korea Soc. Simulat. 26 (1) (2017) 39–46. S. Yu, DSP for image segmentation based on FODPSO algorithm [D], Southwest JiaoTong University, ChengDu, 2016. P. Ghamisi, M.S. Couceiro, J.A. Benediktsson, et al., An efficient method for segmentation of images based on fractional calculus and natural selection [J], Expert Syst. Appl. 39 (16) (2012) 12407–12417. P. Ghamisi, M.S. Couceiro, J.A. Benediktsson, A novel feature selection approach based on FODPSO and SVM [J], IEEE Trans. Geosci. Remote Sens. 53 (5) (2015) 2935–2947. S. Hunziker, S.P. Quanz, A. Amera, A PCA-based approach for subtracting thermal background emission in high-contrast imaging data [J], A & A 29 (9) (2017) 1–12. B. Yousefi, K.L. Chu, Bio-inspired human action recognition using hybrid maxproduct Neuro-Fuzzy classifier and quantum-behaved PSO [J]. arxiv 2015. Y. Bardia, S. Stefano, I.C. Clemente, et al., Thermography data fusion and nonnegative matrix factorization for the evaluation of cultural heritage objects and buildings [J], J. Therm. Anal. Calorim. (2018).