- Email: [email protected]
A novel compression algorithm for infrared thermal image sequence based on K-means method
Infrared Physics & Technology 64 (2014) 18–25
Contents lists available at ScienceDirect
Infrared Physics & Technology journal homepage: www.elsevier.com/locate/infrared
A novel compression algorithm for infrared thermal image sequence based on K-means method Jin-Yu Zhang a,⇑, Wei Xu b, Wei Zhang a, Xiangbin Meng a, Yong Zhang a a b
Xi’an Research Institute of High-Tech, 2 Tongxin Road, Xi’an, 710025, PR China Keppel Offshore & Marine Technology Centre Pte Ltd., 31 Shipyard Road, 628130, Singapore
h i g h l i g h t s A novel global compression algorithm of infrared thermal image sequence is proposed. K-means method is utilized to the compression of the thermal image sequence in space dimension. Thermal image sequences of two embedded defective specimens made of different materials are successfully testified. A high compression ratio is comfortably achieved by exceeding classical TSR algorithm in thousands of times. A reasonable classification number is found.
a r t i c l e
i n f o
Article history: Received 22 October 2013 Available online 31 January 2014 Keywords: Infrared thermal wave image sequence Image compressing and reconstruction K-means method Thermographic inspection
a b s t r a c t High resolution in space and time is becoming the new trend of thermographic inspection of equipments, therefore, the development of a fast and precise processing and data store technique of high resolution thermal image should be well studied. This article will propose a novel global compression algorithm, which will provide an effective way to improve the precision and processing speed of thermal image data. This new algorithm is based on the decay of the temperature of thermograph and the feature of thermal image morphology. Firstly, it will sort the data in space according to K-means method. Then it will employ classic fitting calculation to fit all the typical temperature decay curves. At last, it will use the fitting parameters of the curves as the parameters for compression and reconstruction of thermal image sequence to achieve the method for which the thermal image sequence can be compressed in space and time simultaneously. To validate the proposed new algorithm, the authors used two embedded defective specimens made of different materials to do the experiment. The results show that the proposed infrared thermal image sequence compression processing algorithm is an effective solution with high speed and high precision. Compared to the conventional method, the global compression algorithm is not only noise resistant but also can improve the computing speed in hundreds of times. Ó 2014 Elsevier B.V. All rights reserved.
1. Introduction Infrared thermal wave inspection is a new non-destructive inspection technique which have being applied in more and more engineering projects in recent years [1–5]. The compression processing of infrared thermal image is the critical part since it is the fundamental and premise of all the thermal wave image processing method. And fitting calculation in time domain is the basic image compression method [6–9]. By fitting the infrared thermographic time sequence data, it can reduce the noise of all pixel points in time domain significantly. It also reduces the impact of ⇑ Corresponding author. Address: Xi’an Research Institute of High-Tech, 2 Tongxin Road, Xi’an, Shaanxi 710025, PR China. Tel.: +86 13468822822. E-mail address: [email protected] (J.-Y. Zhang). http://dx.doi.org/10.1016/j.infrared.2014.01.011 1350-4495/Ó 2014 Elsevier B.V. All rights reserved.
uneven heating. Thus it will increase the contrast of defects. Only the fitting coefficients of sequence need to be stored after fitting. These coefficients can be easily applied to the fitting functions to rebuild the original thermal wave images. This can greatly reduce the memory space needed to store the data [7–9]. Since for a set of normal infrared thermal image sequence, it comprises of hundred of thousands or millions of time series. To fit the huge number of time series is a very tough task which needs to spend many hours and makes on site inspection on spot impossible. Therefore, how to compress the image sequence data without losing the graphic features becomes an engineering challenge to be solved urgently. It also has a high value in academic research. Shepard has done a lot of work in thermographic data compression. He proposed a unique method renowned as thermographic sequence reconstruction (TSR) [9]. This technique takes the logarithm
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
of the original data firstly. Secondly, it performs fitting calculation by using polynomial fitting model. Thirdly, it uses parameters of the model to achieve the compressing of image. This technique received good results in real practice. Zhang et al. [10] proposed a non-linear LM fitting method based on thermal wave theory model to replace the polynomial fitting model after an in-depth study of the basis of transmission of thermal wave. This method has achieved good result in some applications. Zhang et al. [11] also proposed a genetic algorithm which gives global optimization performance and accurate result. Some researchers proposed other improving methods. Chen [12] proposed a homotopy alternating iteration method which reduced the dependence on the initial condition like LM algorithm effectively and make convergence in a big range possible. But all these methods have the flaw that they need large computation and long computing hours which impedes the application in real engineering practice. Especially the development of thermal imager in recent years makes the resolution increased from 320 240 to 640 480 or 1024 768. Thus the image sequence to be processed has been increased 4–10 times, as well as time for data fitting and image compression. This makes it more difficult to perform on spot non-destructive inspection for engineering application. This article proposed a global infrared thermal image sequence compression processing method which is based on K-means. Two specimens with defects which are made from different material are used for experiment. The results are compared and studied to validate the proposed method. Section 1 is the description of the proposed global thermal image compression method. Section 2 introduces the experiment and the result analysis. Section 3 is the conclusion. The assessment method of the quality of the images is attached as in Appendix.
2. Thermal image sequence global compressing algorithm The idea of the fast global compressing thermal image sequence method mainly comprises three aspects. Firstly, to class all time series on which the pixel points are located in the thermal wave graph into different categories, at the same time, record the relation between categories and all pixel location, then to select the most representative sequence instead of all the sequences in a category, or to calculate the mean sequence of all sequences in a category, or use the mean sequence to instead of a category. Secondly, to use fitting algorithm to fit all categories of sequences to obtain the parameters of each category sequence, and then pick up the location information of sequence and relevant parameters as the thermal image sequence’s compressing parameter to store. Thirdly, to use the stored data namely the location information, sequence category, fitting model and fitting parameters to reconstruct the thermal image sequence. The first point to discuss is how to class the time series into categories. Time sequence clustering and classifying is a hot research
19
topic in data exploring industry [13,14]. Recently, there are many algorithms have been applied to cluster and classify time series such as decision-making tree, Bayes, artificial neural network, Kmeans and K-Nearest Neighbor. K-means is a mature dynamic clustering method. 2.1. Principle of K-means algorithm and its improvement K-means algorithm is also known as K average algorithm, it is a dynamic clustering method based on distance measure. It is wildly used in scientific research and industry [15–17]. The basic idea of K-means algorithm is: For a given database consisted of number n data sequence objects and number k of classes to be generated, randomly pick up k objects as initial centers of the number k class. Then calculate the distance from the other samples to each class center. Then the sample is classified into the nearest class. Then re-calculate the value of new class center by averaging method after the first classification. Iterate these procedures until the value of any new class center remain unchanged which shows the end of sample adjustment or the criterion function of class mean square error has converged. The sum of square error among all classes is gradually reduced in iteration process, which will converge in an expected error level at last. Although K-means algorithm is a mature clustering method which is fast and effective, its computation performance greatly relies on the selection of the initial class center [17–19]. Since the decrease of the thermal wave temperature follows specific decay rule for which thermal wave theory model is applicable [10], a few representative frames of data can be selected as the feature parameters for the clustering analysis. In order to improve the clustering performance and speed of the algorithm, two measures for improvement are taken. One is to take the average value of the feature parameters as the clustering indicator. This not only reduces the random noise, but also reduces the repeating distance calculation during the clustering process significantly. The other one is to choose initial clustering centers at a specific distance. This helps the clustering algorithm converge quickly since the initial class centers taken in this way are closer to actual ones. The procedures of improved K-means algorithm is described as following: Input of algorithm: number k objects or sequence class. Output of algorithm: number k clusters which gives the minimum square error. (1) For the number n samples to be clustered, divide them equally into k intervals according to their value, take the mid of each interval as the initial class center mi (i = 1, 2, ..., k). (2) Applying Eq. (1) to calculate the distance d(p, mi) from individual sample p to number k class centers.
Fig. 1. Photo of steel shell specimen. (a) The front view. (b) The back view.
20
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
dði; jÞ ¼ jxi xj j
ð1Þ
Here xi and xj are feature parameters. (3) Find the minimum d(p, mi) for each object p, then classify p into the class mi with shortest distance. (4) Repeat (1) to (3) for all the samples, apply Eq. (2) to recalculate the value of class center mi for new classes.
mk ¼
N X xi =N
ð2Þ
i¼1
Here mk is the kth cluster center and N is the number of objects in the kth class. (5) Assign all the objects to the most similar class, then repeat (1) to (4) until minimum sum of the square error E is achieved. The E can be calculated by applying Eq. (3)
E¼
k X X
jp mi j2
ð3Þ
i¼1 p2C i
Here E is the total sum of square error for al objects, p is the object in space, mi is average value of cluster Ci. 2.2. Thermal image global compressing algorithm procedure The procedure for thermal image sequence global compressing based on K-means algorithm is described as the followings:
(4) Calculate the average sequence Xr1, Xr2, ..., Xrl (r = 1, 2, ..., k) in a category for all sequences and then use the average sequence to replace all the corresponding sequences; (5) Use TSR data fitting algorithm to fit class k sequence Xr1, Xr2, ..., Xrl (r = 1, 2, ..., k) to obtain the polynomial parameters ar1, ar2, ..., ar5 (r = 1, 2, ..., k) for each sequence. (6) Save the sequence location information, sequence class and polynomial parameters as the compressing parameters for thermal image sequence to obtain corresponding thermal image sequence compressing document. (7) According to saved sequence class, polynomial model and relevant parameters, etc. to reconstruct the decay curve of the thermal image temperature. (8) According to the location and sequence class information of individual pixel point, restore the sequence of temperature change of each pixel point to initial location and obtain the processed graph data of thermal wave. (9) At last, use the processed thermal wave graph data for final stage of the thermal wave non-destructive inspection. That is image processing, defect qualitative analysis and quantitative analysis. 3. Experiment and result analysis 3.1. Experiment
(1) Since the infrared thermal image sequence is inspected in a fixed view field i.e. the location for inspection remain unchanged, it can be assumed that the resolution of captured thermal image is m n. i.e. m n pixel points and the length of image sequence is l. Read the value of temperature or radiation of each pixel point in pre-defined reading rule (either by column or by row) and record the position information of each pixel point. Number m n time sequences with a length of l which represents the change of temperature of each pixel point can be obtained. (2) Assumed that the change of temperature of individual pixel point can be represented by Xi1, Xi2, ..., Xil (i = 1, 2, ..., m n), pick up the sequence which has the significant difference temperature declining to for the new sequence xi1, xi2, ..., xij (i = 1, 2, ..., m n; j < l). (3) Use K-means clustering and classifying algorithm to process sequence xi1, xi2, ..., xij (i = 1, 2, ..., m n; j < l), the result is represented by C1, C2, ..., Ck, in which k is the number of classes.
Use steel material and glass fiber reinforced plastic envelope paper honeycomb material to make the experimental specimens, then use active infrared thermal wave imaging equipment for thermal wave inspection. The thermal image instrument is the VarioCAM hr research 680 model which has a space resolution of 640 480, a frame rate up to 60 Hz, a spectrum response range from 7.5 lm to 14 lm, a image rate of 50/60 Hz, a temperature measuring range from 40 °C to +1200 °C. It is made by InfraTec Company. It uses pulse heating single side positioning inspection method. Its heating source is two high power flash with a heating power of 4.8 kJ, inspection distance 500 mm from the test piece and the heating pulse can last 2 ms. 3.1.1. Steel shell specimen The specimen has a dimension of 280 mm (length) 200 mm (width) 6 mm (height). There are 8 flat bottom holes on the back side to simulate bonding defects. The four holes on top have a uniform depth of 1 mm and the diameters are 20 mm, 16 mm, 10 mm
Fig. 2. Sketch of defects’ distribution and depth.
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
21
Fig. 3. Typical frame of thermal image sequence of the steel shell specimen. (a) The 21st frame (0.40 s). (b) The 22nd frame (0.42 s). (c) The 31st frame (0.60 s). (d) The 65th frame (1.30 s). (e) The 88th frame (1.76 s). (f) The 245th frame (4.90 s).
Fig. 4. Photo of glass fiber reinforced plastic paper honeycomb material specimen. (a) The front view. (b) The back view.
Fig. 5. The typical frames of the glass fiber reinforced paper honeycomb material specimen. (a) The 50th frame (0.98 s). (b) The 150th frame (2.98 s). (c) The 200th frame (3.98 s). (d) The 270th frame (5.38 s).
22
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
Initial image
Initial image
100 classes
50 classes
150 classes
100 classes
200 classes
150 classes Fig. 7. Comparison between before and after the global compressing process of the thermal image for glass fiber reinforced plastic paper honeycomb material specimen. (a) The 150th frame. (b) The 200th frame.
Fig. 6. Comparison between before and after the global compressing process of the thermal image for steel shell specimen. (a) The 31st frame. (b) The 65th frame.
and 5 mm respectively; the four holes at the bottom have a uniform diameter of 20 mm and the depths are 2 mm, 3 mm, 4 mm, 5 mm respectively. Photos of the specimen have been taken which are shown in Figs. 1 and 2. Fig. 1(a) shows the front view of the specimen while Fig. 1(b) shows the back view of the specimen. Fig. 2 is the schematic description of the defects’ distribution and depth. The property of the specimen is defined as: thermal conductivity k = 36.7 W/(m K), specific heat capacity c = 460 J/(kg °C), and density q = 7800 kg/m3. The two flash lights have a heating power of 2.4 KJ, and the image collection frequency is 50 Hz. For a collection time of 5.1 s, 256 frames of images will be collected. Fig. 3 shows the typical single frame image in the thermal wave image sequence of the specimen, from which it can be seen that the defect with bigger diameter and less depth will appear first. After some time, the one has smaller diameter and greater depth starts to appear and becomes clearer. This change takes about 0.6 s; meanwhile, the heat spot has the most contrast with surrounding environment. After the peak, the contrast starts to decline gradually. After 1.76 s, the heat spot of the defect with smallest diameter disappears first; and all the defects’ hot spot almost completely disappear after about 4.90 s. The surface temperature tends to be even.
3.1.2. Glass fiber reinforced plastic envelope paper honeycomb material specimen The specimen has a dimension of 300 mm (length) 150 mm (width) 5 mm (height). For the height, the thickness of the glass fiber reinforced plastic skin which is 1 mm at both sides, and the thickness of paper honeycomb in the middle is 3 mm. There are four circular defects purposely made on top of the specimen to simulate the debonding between the plastic layer and the core
layer. The diameters of the four defects are 20 mm, 15 mm, 10 mm and 5 mm respectively. The picture of the actual test piece is shown in Fig. 4, in which Fig. 4(a) is the front view and Fig. 4(b) is the back view. The two flashes have a heating power of 2.4 KJ and the image collection frequency is 50 Hz. The capture time is 5.98 s, and total 300 frames of images are collected. Fig. 5 shows the typical frames of the thermal wave image sequence of the glass fiber reinforced paper honeycomb material specimen. It can be seen that all four defects appear at the same time due to they have the same to distance to the surface of the test piece. As time goes on, a new thermal balance has been built gradually. It is necessary to note that the bright area in center below is due to the reflection of flash rather than a defective area and the right corner dark area is due to thermal isolation effect caused by paper label.
3.2. Experimental result and analysis 10 Image frames with good display effect are selected from the thermal wave image sequences of the two specimens respectively. They are used to make the classified image sequence (image frame 31–40 are selected for steel shell specimen while image frame 141–150 are selected for glass fiber reinforced plastic paper honeycomb material specimen). Follow the procedures described in Section 1.2 to perform classification, and then go thru replacement and image restoring to obtain the results which are shown in Figs. 6 and 7. The thermographic sequences of both specimens are processed in different number of classes namely 50, 100 and 150. In experiment, we have produced a lot of thermal images. However, it is impossible to show all in this paper. We selected two representative thermal images for each specimen i.e. a sharp one and a normal one to demonstrate the results. For the steel specimen, a sharp image of the 31st frame and a normal one of the 65th frame are selected. While for the glass fiber reinforced paper honeycomb
23
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
a
b
Fig. 8. Comparison between the initial images and the reconstructed images from the 31st frame thermal image of specimen 1. (a) Original thermal image. (b) Reconstructed thermal image.
Table 1 The evaluation of the image quality after global compressing process for specimen 1. Classes
Initial image 50 Classes 100 Classes 150 Classes
Entropy
Space frequency
RMSE
PSNR
Frame 31
Frame 65
Frame 31
Frame 65
Frame 31
Frame 65
Frame 31
Frame 65
6.2270 5.0690 5.4754 5.9790
6.4293 5.1524 5.7387 5.8680
4.6010 7.7583 5.8817 5.8345
4.8550 11.7703 10.4005 10.0654
0 0.0220 0.0157 0.0157
0 0.0283 0.0504 0.0346
Inf. 187.1238 193.8533 193.8533
Inf. 182.0975 170.5903 178.0841
Table 2 The evaluation of the image quality after global compressing process for specimen 2. Classes
Initial image 100 Classes 150 Classes 200 Classes
Entropy
Space frequency
RMSE
PSNR
Frame 00
Frame 200
Frame 00
Frame 200
Frame 00
Frame 200
Frame 00
Frame 200
6.3414 5.9263 6.1106 6.1133
6.2408 5.7021 5.8588 5.8634
3.9966 5.1060 4.8885 4.6878
3.8182 9.0974 8.2914 8.0732
0 0.0661 0.0661 0.0661
0 0.0630 0.0630 0.0630
Inf. 165.1516 165.1516 165.1516
Inf. 166.1274 166.1274 166.1274
Table 3 Comparison of compression computation time. Name
Number of classes
TSR
Proposed method
Specimen 1 Specimen 2
150 200
37.6 s 45.4 s
29.3 s 32.1 s
specimen, a sharp image of the 150th frame and a normal one of the 200th frame are selected. They are shown in Figs. 6 and 7. In order to observe the difference between the initial images and the reconstructed images, a 3D plot of the important parameters from quantitative analysis is shown in Fig. 8 which used the data from the 31st frame thermal image of the first specimen. From the results shown in Figs. 6 and 7, it can be seen that result obtained by using K-means clustering and classification method to classify the original thermal image data and then restore the thermal image form the classified data has achieved an ideal effect.
The result has kept the important information for the feature of the defects. From Figs. 6 and 8, it can be seen that the display of the boundary of part of the restored defects are more distinct, with a more regular shape, and this will facilitate the quantitative identification of the size of the defects. Meanwhile, it also can be seen that the image resolution was slightly reduced after processing and obvious stratification of the display of the images occurred. This is because the number of classes is relatively less compared to original data. However, it does not affect the display effect of the defects. Although the resolution of the reconstructed image will be higher and display effect will be better if there were more classes in theory, it is found in experiment that the processing effect would not change when K-means classes exceeded 150–200. This is because that a thermal image is a BMP image with a grayscale of 8 bit which has a maximum level of 256, while human being’s eyes only can sense a gray level of 100. Thus when using classes more than 200, it is hardly found any difference by naked eye’s observation, and the indicators of quantitative analysis also shows little change. Besides, due to bigger amount of image
24
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
sequence data, the program posed higher requirement of the RAM space during execution. The more classes, the more RAM are needed. The hardware might become a constraint to support too many classes. Tables 1 and 2 separately listed the value of assessed parameters of the image quality after global compressing process of the thermal images for two specimens. It can be seen from the table that the value of entropy increased gradually when the number of class increased and approached the original value of entropy. But the space frequency of the image decreased gradually and it kept being greater than value of original thermal image all the way. This shows that the detail of the image after classification is richer than that of original thermal image and defective area has a clearer display. It also clears the misunderstanding that the more classes the better display effect of the defects. Moreover, it can be seen from the table that Root-Mean-Square error became smaller after classification and peak value of signal-to-noise ratio became greater. This shows that the error between processed image and original one is reasonably small and the processing effect is acceptably good. Refer to Table 2, although the number of class is different, two parameters namely RMSE and PSNR are exactly the same. This shows that the difference between processed thermal image and original image remains unchanged. This further proves that the effect of processing is not directly dependent on the number of classes. The significance of classification is to use the least data to substitute the large amount original data with a basis of keeping the important feature information of defects. Take the steel shell specimen as an example, the resolution of collected image in the experiment is 359 281 and the number of sampling frames is 256. Therefore the original thermal image sequence includes 100,879 time sequences with a length of 256 which is a very large amount of data. But the amount of data can be significantly reduced after classification processing. If 100 classes were adopted, it only needs 100 sequences to substitute the original 100,879 sequences. The space compressing ratio of data compressing reaches 1008:1 and the compressing efficiency is very high with keeping the feature information of the original thermal images. Additionally, in the later fitting processing, it only needs to fit 100 sequences compared to the original 100,879 ones. This can save great processing time and significantly increase the fitting efficiency. Furthermore, there is to gain compressing benefit compared using other data fitting like TSR. For example, a typical thermal image sequence with 640 pixel 480 pixel 1000 frames can be classified into 150 classed. The global compressing algorithm has a time compressing ratio of 167:1, a space compressing ratio is 2048:1 and a global compressing ratio of 342,016:1. A thermal image sequence original data file is several hundred MB in its size may have a compressing file with only few KB. It is much smaller than that of TSR algorithm with a size over ten MB. For the time spent in thermal image compression computation, it is not applicable to compare iteration fitting algorithm to TSR fitting. Because TSR method computes directly the polynomial to fit a data sequence, while the algorithms presented in [10–12] are iteration fitting methods that need to go through 6 to dozens of times computation loops to do the same work. Although the speed of iteration fitting algorithm has been optimized, it is still dozens or hundred’s times slower than TSR fitting. This method proposed in this paper has significantly reduced the computation task by reducing 99.95–99.9% of the numbers of image sequences to be fitted, notwithstanding an increase of the K-means clustering calculation time. We used previous two sets of thermal image data to compare the computer processing time of the proposed method and that of TSR. The result is shown in Table 3.
The configuration of the computer: Intel Pentium P6200 CPU, dual cores, 2.13 GHz; 2G RAM; 320G hard disk; Windows XP OS; application programs developed in Delphi7. From Table 3, it can be seen that the proposed method offers faster speed than classic TSR method. Hereby we selected the data with largest class number for the comparison for which the compression effect can be considered the same for using both methods. 4. Conclusions All above, the study of K-mean algorithm based thermal wave image sequence global compressing algorithm can draw the conclusion as following. (1) For specimens made from different materials, provided a clear original thermal image can be obtained, this method can achieve an accurate classification of thermal image data. (2) This method not only keeps the original thermal image features, but also can classify and replace the data in a good order. For this experiment, the data space compressing ratio reached 1008:1 after processing and overall compressing ratio reached 40,000:1. This greatly saved storage space and consequential post processing computing workload. (3) In theory, the more classes, the closer the restored thermal image to original one. However, the experiment showed it a misunderstanding. It is necessary to set an appropriate number of classes to obtain optimum result. Normally, it is recommended to use 100–200 classes which could lead to optimum result.
Acknowledgments This work is supported by the National Natural Science Foundation of China (Grant Nos. 51275518 and 51305447) and the Natural Science Foundation of Shaanxi Province (Grant No. 2013JM7021). Appendix A In image data processing field, people are devoted to keep finding the way to assess the quality of an image after its processing. However, there is no uniform assessment standard up to date. The existing assessment methods are generally divided into two namely subjective ones and objective ones. A.1. Subjective assessment method For subjective assessment, an observer will judge on the quality of the visual effect of a testing image and mark it according to a set of pre-defined assessment criterion or his/her experience. Although this method allows the observer to make good use his/ her experience and reflects the intuitionistic quality well, it ignores quantitative measurement and not good for information processing due to lack of mathematical model description. And the result of assessment is limited by the observer’s knowledge background, emotion and degree of fatigue, etc. Subjective assessment method has limited application in real practice. A.2. Objective assessment method The advantage of objective assessment method is that it allows to perform quantitative calculation and measurement on the experimental images based on specific mathematical model to obtain information including luminance, contrast ratio, entropy, Root-Mean-Square error, peak value signal to noise ratio, spatial
25
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
frequency, etc. In order to assess the quality of the processed image more accurately, this article compared four objective assessment parameters namely entropy, spatial frequency, Root-Mean-Square error and peak value signal to noise ratio. A brief introduction of the four parameters is as following: (1) Entropy is from information entropy theory. The more uncertain the variable is, the greater the value of entropy. Information entropy is an important parameter to measure the quantity of information in information theory. If a system was more unordered, the value of information entropy is greater. The definition of information entropy when it is applied to objective assessment of image is as following: N X H ¼ ðEi ðx; yÞlog2 Ei ðx; yÞÞ
ðA1Þ
i¼1
where E(x, y) is the occurring frequency of the grey level of any individual pixel point in the whole image, log 2Ei(x, y) is the corresponding logarithm of E(x, y). For any image, the greater the image’s information entropy is, the larger amount of information and the more details the image possesses. (2) Spatial frequency (SF) can represent the clearness level of an image. It is an important indication which is used to measure the level of information richness of the image. The higher the thermal wave image’s spatial frequency is, the richer the detail of the image is. The definition of spatial frequency is as following.
SF ¼
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi RF2 þ CF2
ðA2Þ
When RF is row spatial frequency, CF is column spatial frequency
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u M X N u 1 X RF ¼ t ½Fðm; nÞ Fðm; n 1Þ2 MN m¼1 n¼2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N X M u 1 X CF ¼ t ½Fðm; nÞ Fðm 1; nÞ2 MN n¼1 m¼2
ðA3Þ
ðA4Þ
where F(m, n) represents the grey level of a pixel point located at (m, n) in coordinate. (3) Root-Mean-Square-Error (RMSE) is the error between processed image F and standard reference image R.
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi PM PN 2 i¼1 j¼1 ðRði; jÞ Fði; jÞÞ RMSE ¼ MN
ðA5Þ
RMSE reflects the level of difference between processed image and standard reference image. Smaller value of RMSE shows that the change is small after applying enhancement algorithm. In order to facilitate the calculation, peak value signal to noise ratio which is
inverse proportional to Root-Mean-Square error can be taken as the calculation indication. (4) Peak value signal to noise ratio (PSNR) is defined as following:
PSNR ¼ 10 log
255 255 RMSE2
ðA6Þ
The higher the value of PSNR is, the better the processing effect of the image is.
References [1] M. Genest, M. Martinez, N. Mrad, Pulsed thermography for non-destructive evaluation and damage growth monitoring of bonded repairs, Compos. Struct. 88 (1) (2009) 112–120. [2] M.R. Valluzzi, E. Grinzato, C. Pellegrino, IR thermography for interface analysis of FRP laminates externally bonded to RC beams, Mater. Struct. 42 (1) (2009) 25–34. [3] V. Crupi, G. Chiofalo, E. Guglielmino, Using infrared thermography in low cycle fatigue studies of welded joints, Welding J. 89 (9) (2010) 195–200. [4] R. Jones, M. Krishnapillai, K. Cairns, Application of infrared thermography to study crack growth and fatigue life extension procedures, Fatigue Fract. Eng. Mater. Struct. 33 (12) (2010) 871–884. [5] T. Hannes, G. Stefan, Pulsed thermography of CFC monoblock divertor components, Fusion Eng. Des. 84 (11) (2009) 1867–1870. [6] X.L. Yang, T. Jang, L.C. Feng, Thermographic testing for impact damage of airplane composite, Nondestruct. Test. 31 (2) (2009) 120–122. [7] Y.Y. Zhao, Y.P. Tian, Data fitting method in pulsed phase thermography, Nondestruct. Test. 29 (11) (2007) 637–640. [8] X.W. Guo, V. Vavilov, V. Shiryaev, Thermal nondestructive testing of corrosion in aviation aluminum panels and data processing algorithms, Chinese J. Mech. Eng. 45 (3) (2009) 208–213. [9] S.M. Shepard, T. Ahmed, B.A. Rubadeux, D. Wang, J.R. Lhota, Synthetic processing of pulsed thermographic data for inspection of turbine components, Insight Brit. Inst. NDT 43 (9) (2001) 587–589. [10] Y. Zhang, J.Y. Zhang, J.X. Huang, Infrared thermal imaging data fitting method based on theoretical model of infrared thermal detection, Infrared 33 (4) (2012) 38–41. [11] Y. Zhang, J.Y. Zhang, X.R. Huang, J.X. Huang, The infrared thermal imaging data fitting method based on genetic algorithm, Nondestruct. Test. 34 (10) (2012) 38–41. [12] H. Chen, S.G. Deng, Y.R. Fan, Application of homotopy alternative iteration method in double exponential fitting, Comput. Eng. Appl. 43 (25) (2007) 204– 205. [13] Z. Lin, Y. Xing, Survey of feature extraction approaches for time series classification, Comput. Syst. Appl. 21 (10) (2012) 224–229. [14] L.L. Li, A review on classification algorithms in data mining, J. Chongqing Normal Univ. (Nat. Sci.) 28 (4) (2011) 44–47. [15] W. Zhen, M.Q. Tong, The biological and physical mechanism of infrared thermal imaging in medical diagnosis, Life Sci. Instr. 3 (2) (2005) 20–22. [16] H. Jiang, R. Zhang, Key technologies in registration and fusion for infrared and visible images, Infrared Laser Eng. 35 (10) (2006) 7–12. [17] W. Zhao, M. Zhang, W.H. Li, Improved K-means clustering algorithm on space, Appl. Res. Comp. 25 (7) (2008) 1995–1997. [18] T. Huang, S.H. Liu, Y.N. Tan, Research of clustering algorithm based on Kmeans, Comp. Technol. Dev. 21 (7) (2011) 54–57. [19] Shokri Z. Selem, M.A. Imail, K-means-type algorithms: a generalized convergence theorem and characterization of local optimality, IEEE Trans. Pattern Anal. Mach. Int. 6 (1) (1984) 5–50.
Contents lists available at ScienceDirect
Infrared Physics & Technology journal homepage: www.elsevier.com/locate/infrared
A novel compression algorithm for infrared thermal image sequence based on K-means method Jin-Yu Zhang a,⇑, Wei Xu b, Wei Zhang a, Xiangbin Meng a, Yong Zhang a a b
Xi’an Research Institute of High-Tech, 2 Tongxin Road, Xi’an, 710025, PR China Keppel Offshore & Marine Technology Centre Pte Ltd., 31 Shipyard Road, 628130, Singapore
h i g h l i g h t s A novel global compression algorithm of infrared thermal image sequence is proposed. K-means method is utilized to the compression of the thermal image sequence in space dimension. Thermal image sequences of two embedded defective specimens made of different materials are successfully testified. A high compression ratio is comfortably achieved by exceeding classical TSR algorithm in thousands of times. A reasonable classification number is found.
a r t i c l e
i n f o
Article history: Received 22 October 2013 Available online 31 January 2014 Keywords: Infrared thermal wave image sequence Image compressing and reconstruction K-means method Thermographic inspection
a b s t r a c t High resolution in space and time is becoming the new trend of thermographic inspection of equipments, therefore, the development of a fast and precise processing and data store technique of high resolution thermal image should be well studied. This article will propose a novel global compression algorithm, which will provide an effective way to improve the precision and processing speed of thermal image data. This new algorithm is based on the decay of the temperature of thermograph and the feature of thermal image morphology. Firstly, it will sort the data in space according to K-means method. Then it will employ classic fitting calculation to fit all the typical temperature decay curves. At last, it will use the fitting parameters of the curves as the parameters for compression and reconstruction of thermal image sequence to achieve the method for which the thermal image sequence can be compressed in space and time simultaneously. To validate the proposed new algorithm, the authors used two embedded defective specimens made of different materials to do the experiment. The results show that the proposed infrared thermal image sequence compression processing algorithm is an effective solution with high speed and high precision. Compared to the conventional method, the global compression algorithm is not only noise resistant but also can improve the computing speed in hundreds of times. Ó 2014 Elsevier B.V. All rights reserved.
1. Introduction Infrared thermal wave inspection is a new non-destructive inspection technique which have being applied in more and more engineering projects in recent years [1–5]. The compression processing of infrared thermal image is the critical part since it is the fundamental and premise of all the thermal wave image processing method. And fitting calculation in time domain is the basic image compression method [6–9]. By fitting the infrared thermographic time sequence data, it can reduce the noise of all pixel points in time domain significantly. It also reduces the impact of ⇑ Corresponding author. Address: Xi’an Research Institute of High-Tech, 2 Tongxin Road, Xi’an, Shaanxi 710025, PR China. Tel.: +86 13468822822. E-mail address: [email protected] (J.-Y. Zhang). http://dx.doi.org/10.1016/j.infrared.2014.01.011 1350-4495/Ó 2014 Elsevier B.V. All rights reserved.
uneven heating. Thus it will increase the contrast of defects. Only the fitting coefficients of sequence need to be stored after fitting. These coefficients can be easily applied to the fitting functions to rebuild the original thermal wave images. This can greatly reduce the memory space needed to store the data [7–9]. Since for a set of normal infrared thermal image sequence, it comprises of hundred of thousands or millions of time series. To fit the huge number of time series is a very tough task which needs to spend many hours and makes on site inspection on spot impossible. Therefore, how to compress the image sequence data without losing the graphic features becomes an engineering challenge to be solved urgently. It also has a high value in academic research. Shepard has done a lot of work in thermographic data compression. He proposed a unique method renowned as thermographic sequence reconstruction (TSR) [9]. This technique takes the logarithm
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
of the original data firstly. Secondly, it performs fitting calculation by using polynomial fitting model. Thirdly, it uses parameters of the model to achieve the compressing of image. This technique received good results in real practice. Zhang et al. [10] proposed a non-linear LM fitting method based on thermal wave theory model to replace the polynomial fitting model after an in-depth study of the basis of transmission of thermal wave. This method has achieved good result in some applications. Zhang et al. [11] also proposed a genetic algorithm which gives global optimization performance and accurate result. Some researchers proposed other improving methods. Chen [12] proposed a homotopy alternating iteration method which reduced the dependence on the initial condition like LM algorithm effectively and make convergence in a big range possible. But all these methods have the flaw that they need large computation and long computing hours which impedes the application in real engineering practice. Especially the development of thermal imager in recent years makes the resolution increased from 320 240 to 640 480 or 1024 768. Thus the image sequence to be processed has been increased 4–10 times, as well as time for data fitting and image compression. This makes it more difficult to perform on spot non-destructive inspection for engineering application. This article proposed a global infrared thermal image sequence compression processing method which is based on K-means. Two specimens with defects which are made from different material are used for experiment. The results are compared and studied to validate the proposed method. Section 1 is the description of the proposed global thermal image compression method. Section 2 introduces the experiment and the result analysis. Section 3 is the conclusion. The assessment method of the quality of the images is attached as in Appendix.
2. Thermal image sequence global compressing algorithm The idea of the fast global compressing thermal image sequence method mainly comprises three aspects. Firstly, to class all time series on which the pixel points are located in the thermal wave graph into different categories, at the same time, record the relation between categories and all pixel location, then to select the most representative sequence instead of all the sequences in a category, or to calculate the mean sequence of all sequences in a category, or use the mean sequence to instead of a category. Secondly, to use fitting algorithm to fit all categories of sequences to obtain the parameters of each category sequence, and then pick up the location information of sequence and relevant parameters as the thermal image sequence’s compressing parameter to store. Thirdly, to use the stored data namely the location information, sequence category, fitting model and fitting parameters to reconstruct the thermal image sequence. The first point to discuss is how to class the time series into categories. Time sequence clustering and classifying is a hot research
19
topic in data exploring industry [13,14]. Recently, there are many algorithms have been applied to cluster and classify time series such as decision-making tree, Bayes, artificial neural network, Kmeans and K-Nearest Neighbor. K-means is a mature dynamic clustering method. 2.1. Principle of K-means algorithm and its improvement K-means algorithm is also known as K average algorithm, it is a dynamic clustering method based on distance measure. It is wildly used in scientific research and industry [15–17]. The basic idea of K-means algorithm is: For a given database consisted of number n data sequence objects and number k of classes to be generated, randomly pick up k objects as initial centers of the number k class. Then calculate the distance from the other samples to each class center. Then the sample is classified into the nearest class. Then re-calculate the value of new class center by averaging method after the first classification. Iterate these procedures until the value of any new class center remain unchanged which shows the end of sample adjustment or the criterion function of class mean square error has converged. The sum of square error among all classes is gradually reduced in iteration process, which will converge in an expected error level at last. Although K-means algorithm is a mature clustering method which is fast and effective, its computation performance greatly relies on the selection of the initial class center [17–19]. Since the decrease of the thermal wave temperature follows specific decay rule for which thermal wave theory model is applicable [10], a few representative frames of data can be selected as the feature parameters for the clustering analysis. In order to improve the clustering performance and speed of the algorithm, two measures for improvement are taken. One is to take the average value of the feature parameters as the clustering indicator. This not only reduces the random noise, but also reduces the repeating distance calculation during the clustering process significantly. The other one is to choose initial clustering centers at a specific distance. This helps the clustering algorithm converge quickly since the initial class centers taken in this way are closer to actual ones. The procedures of improved K-means algorithm is described as following: Input of algorithm: number k objects or sequence class. Output of algorithm: number k clusters which gives the minimum square error. (1) For the number n samples to be clustered, divide them equally into k intervals according to their value, take the mid of each interval as the initial class center mi (i = 1, 2, ..., k). (2) Applying Eq. (1) to calculate the distance d(p, mi) from individual sample p to number k class centers.
Fig. 1. Photo of steel shell specimen. (a) The front view. (b) The back view.
20
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
dði; jÞ ¼ jxi xj j
ð1Þ
Here xi and xj are feature parameters. (3) Find the minimum d(p, mi) for each object p, then classify p into the class mi with shortest distance. (4) Repeat (1) to (3) for all the samples, apply Eq. (2) to recalculate the value of class center mi for new classes.
mk ¼
N X xi =N
ð2Þ
i¼1
Here mk is the kth cluster center and N is the number of objects in the kth class. (5) Assign all the objects to the most similar class, then repeat (1) to (4) until minimum sum of the square error E is achieved. The E can be calculated by applying Eq. (3)
E¼
k X X
jp mi j2
ð3Þ
i¼1 p2C i
Here E is the total sum of square error for al objects, p is the object in space, mi is average value of cluster Ci. 2.2. Thermal image global compressing algorithm procedure The procedure for thermal image sequence global compressing based on K-means algorithm is described as the followings:
(4) Calculate the average sequence Xr1, Xr2, ..., Xrl (r = 1, 2, ..., k) in a category for all sequences and then use the average sequence to replace all the corresponding sequences; (5) Use TSR data fitting algorithm to fit class k sequence Xr1, Xr2, ..., Xrl (r = 1, 2, ..., k) to obtain the polynomial parameters ar1, ar2, ..., ar5 (r = 1, 2, ..., k) for each sequence. (6) Save the sequence location information, sequence class and polynomial parameters as the compressing parameters for thermal image sequence to obtain corresponding thermal image sequence compressing document. (7) According to saved sequence class, polynomial model and relevant parameters, etc. to reconstruct the decay curve of the thermal image temperature. (8) According to the location and sequence class information of individual pixel point, restore the sequence of temperature change of each pixel point to initial location and obtain the processed graph data of thermal wave. (9) At last, use the processed thermal wave graph data for final stage of the thermal wave non-destructive inspection. That is image processing, defect qualitative analysis and quantitative analysis. 3. Experiment and result analysis 3.1. Experiment
(1) Since the infrared thermal image sequence is inspected in a fixed view field i.e. the location for inspection remain unchanged, it can be assumed that the resolution of captured thermal image is m n. i.e. m n pixel points and the length of image sequence is l. Read the value of temperature or radiation of each pixel point in pre-defined reading rule (either by column or by row) and record the position information of each pixel point. Number m n time sequences with a length of l which represents the change of temperature of each pixel point can be obtained. (2) Assumed that the change of temperature of individual pixel point can be represented by Xi1, Xi2, ..., Xil (i = 1, 2, ..., m n), pick up the sequence which has the significant difference temperature declining to for the new sequence xi1, xi2, ..., xij (i = 1, 2, ..., m n; j < l). (3) Use K-means clustering and classifying algorithm to process sequence xi1, xi2, ..., xij (i = 1, 2, ..., m n; j < l), the result is represented by C1, C2, ..., Ck, in which k is the number of classes.
Use steel material and glass fiber reinforced plastic envelope paper honeycomb material to make the experimental specimens, then use active infrared thermal wave imaging equipment for thermal wave inspection. The thermal image instrument is the VarioCAM hr research 680 model which has a space resolution of 640 480, a frame rate up to 60 Hz, a spectrum response range from 7.5 lm to 14 lm, a image rate of 50/60 Hz, a temperature measuring range from 40 °C to +1200 °C. It is made by InfraTec Company. It uses pulse heating single side positioning inspection method. Its heating source is two high power flash with a heating power of 4.8 kJ, inspection distance 500 mm from the test piece and the heating pulse can last 2 ms. 3.1.1. Steel shell specimen The specimen has a dimension of 280 mm (length) 200 mm (width) 6 mm (height). There are 8 flat bottom holes on the back side to simulate bonding defects. The four holes on top have a uniform depth of 1 mm and the diameters are 20 mm, 16 mm, 10 mm
Fig. 2. Sketch of defects’ distribution and depth.
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
21
Fig. 3. Typical frame of thermal image sequence of the steel shell specimen. (a) The 21st frame (0.40 s). (b) The 22nd frame (0.42 s). (c) The 31st frame (0.60 s). (d) The 65th frame (1.30 s). (e) The 88th frame (1.76 s). (f) The 245th frame (4.90 s).
Fig. 4. Photo of glass fiber reinforced plastic paper honeycomb material specimen. (a) The front view. (b) The back view.
Fig. 5. The typical frames of the glass fiber reinforced paper honeycomb material specimen. (a) The 50th frame (0.98 s). (b) The 150th frame (2.98 s). (c) The 200th frame (3.98 s). (d) The 270th frame (5.38 s).
22
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
Initial image
Initial image
100 classes
50 classes
150 classes
100 classes
200 classes
150 classes Fig. 7. Comparison between before and after the global compressing process of the thermal image for glass fiber reinforced plastic paper honeycomb material specimen. (a) The 150th frame. (b) The 200th frame.
Fig. 6. Comparison between before and after the global compressing process of the thermal image for steel shell specimen. (a) The 31st frame. (b) The 65th frame.
and 5 mm respectively; the four holes at the bottom have a uniform diameter of 20 mm and the depths are 2 mm, 3 mm, 4 mm, 5 mm respectively. Photos of the specimen have been taken which are shown in Figs. 1 and 2. Fig. 1(a) shows the front view of the specimen while Fig. 1(b) shows the back view of the specimen. Fig. 2 is the schematic description of the defects’ distribution and depth. The property of the specimen is defined as: thermal conductivity k = 36.7 W/(m K), specific heat capacity c = 460 J/(kg °C), and density q = 7800 kg/m3. The two flash lights have a heating power of 2.4 KJ, and the image collection frequency is 50 Hz. For a collection time of 5.1 s, 256 frames of images will be collected. Fig. 3 shows the typical single frame image in the thermal wave image sequence of the specimen, from which it can be seen that the defect with bigger diameter and less depth will appear first. After some time, the one has smaller diameter and greater depth starts to appear and becomes clearer. This change takes about 0.6 s; meanwhile, the heat spot has the most contrast with surrounding environment. After the peak, the contrast starts to decline gradually. After 1.76 s, the heat spot of the defect with smallest diameter disappears first; and all the defects’ hot spot almost completely disappear after about 4.90 s. The surface temperature tends to be even.
3.1.2. Glass fiber reinforced plastic envelope paper honeycomb material specimen The specimen has a dimension of 300 mm (length) 150 mm (width) 5 mm (height). For the height, the thickness of the glass fiber reinforced plastic skin which is 1 mm at both sides, and the thickness of paper honeycomb in the middle is 3 mm. There are four circular defects purposely made on top of the specimen to simulate the debonding between the plastic layer and the core
layer. The diameters of the four defects are 20 mm, 15 mm, 10 mm and 5 mm respectively. The picture of the actual test piece is shown in Fig. 4, in which Fig. 4(a) is the front view and Fig. 4(b) is the back view. The two flashes have a heating power of 2.4 KJ and the image collection frequency is 50 Hz. The capture time is 5.98 s, and total 300 frames of images are collected. Fig. 5 shows the typical frames of the thermal wave image sequence of the glass fiber reinforced paper honeycomb material specimen. It can be seen that all four defects appear at the same time due to they have the same to distance to the surface of the test piece. As time goes on, a new thermal balance has been built gradually. It is necessary to note that the bright area in center below is due to the reflection of flash rather than a defective area and the right corner dark area is due to thermal isolation effect caused by paper label.
3.2. Experimental result and analysis 10 Image frames with good display effect are selected from the thermal wave image sequences of the two specimens respectively. They are used to make the classified image sequence (image frame 31–40 are selected for steel shell specimen while image frame 141–150 are selected for glass fiber reinforced plastic paper honeycomb material specimen). Follow the procedures described in Section 1.2 to perform classification, and then go thru replacement and image restoring to obtain the results which are shown in Figs. 6 and 7. The thermographic sequences of both specimens are processed in different number of classes namely 50, 100 and 150. In experiment, we have produced a lot of thermal images. However, it is impossible to show all in this paper. We selected two representative thermal images for each specimen i.e. a sharp one and a normal one to demonstrate the results. For the steel specimen, a sharp image of the 31st frame and a normal one of the 65th frame are selected. While for the glass fiber reinforced paper honeycomb
23
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
a
b
Fig. 8. Comparison between the initial images and the reconstructed images from the 31st frame thermal image of specimen 1. (a) Original thermal image. (b) Reconstructed thermal image.
Table 1 The evaluation of the image quality after global compressing process for specimen 1. Classes
Initial image 50 Classes 100 Classes 150 Classes
Entropy
Space frequency
RMSE
PSNR
Frame 31
Frame 65
Frame 31
Frame 65
Frame 31
Frame 65
Frame 31
Frame 65
6.2270 5.0690 5.4754 5.9790
6.4293 5.1524 5.7387 5.8680
4.6010 7.7583 5.8817 5.8345
4.8550 11.7703 10.4005 10.0654
0 0.0220 0.0157 0.0157
0 0.0283 0.0504 0.0346
Inf. 187.1238 193.8533 193.8533
Inf. 182.0975 170.5903 178.0841
Table 2 The evaluation of the image quality after global compressing process for specimen 2. Classes
Initial image 100 Classes 150 Classes 200 Classes
Entropy
Space frequency
RMSE
PSNR
Frame 00
Frame 200
Frame 00
Frame 200
Frame 00
Frame 200
Frame 00
Frame 200
6.3414 5.9263 6.1106 6.1133
6.2408 5.7021 5.8588 5.8634
3.9966 5.1060 4.8885 4.6878
3.8182 9.0974 8.2914 8.0732
0 0.0661 0.0661 0.0661
0 0.0630 0.0630 0.0630
Inf. 165.1516 165.1516 165.1516
Inf. 166.1274 166.1274 166.1274
Table 3 Comparison of compression computation time. Name
Number of classes
TSR
Proposed method
Specimen 1 Specimen 2
150 200
37.6 s 45.4 s
29.3 s 32.1 s
specimen, a sharp image of the 150th frame and a normal one of the 200th frame are selected. They are shown in Figs. 6 and 7. In order to observe the difference between the initial images and the reconstructed images, a 3D plot of the important parameters from quantitative analysis is shown in Fig. 8 which used the data from the 31st frame thermal image of the first specimen. From the results shown in Figs. 6 and 7, it can be seen that result obtained by using K-means clustering and classification method to classify the original thermal image data and then restore the thermal image form the classified data has achieved an ideal effect.
The result has kept the important information for the feature of the defects. From Figs. 6 and 8, it can be seen that the display of the boundary of part of the restored defects are more distinct, with a more regular shape, and this will facilitate the quantitative identification of the size of the defects. Meanwhile, it also can be seen that the image resolution was slightly reduced after processing and obvious stratification of the display of the images occurred. This is because the number of classes is relatively less compared to original data. However, it does not affect the display effect of the defects. Although the resolution of the reconstructed image will be higher and display effect will be better if there were more classes in theory, it is found in experiment that the processing effect would not change when K-means classes exceeded 150–200. This is because that a thermal image is a BMP image with a grayscale of 8 bit which has a maximum level of 256, while human being’s eyes only can sense a gray level of 100. Thus when using classes more than 200, it is hardly found any difference by naked eye’s observation, and the indicators of quantitative analysis also shows little change. Besides, due to bigger amount of image
24
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
sequence data, the program posed higher requirement of the RAM space during execution. The more classes, the more RAM are needed. The hardware might become a constraint to support too many classes. Tables 1 and 2 separately listed the value of assessed parameters of the image quality after global compressing process of the thermal images for two specimens. It can be seen from the table that the value of entropy increased gradually when the number of class increased and approached the original value of entropy. But the space frequency of the image decreased gradually and it kept being greater than value of original thermal image all the way. This shows that the detail of the image after classification is richer than that of original thermal image and defective area has a clearer display. It also clears the misunderstanding that the more classes the better display effect of the defects. Moreover, it can be seen from the table that Root-Mean-Square error became smaller after classification and peak value of signal-to-noise ratio became greater. This shows that the error between processed image and original one is reasonably small and the processing effect is acceptably good. Refer to Table 2, although the number of class is different, two parameters namely RMSE and PSNR are exactly the same. This shows that the difference between processed thermal image and original image remains unchanged. This further proves that the effect of processing is not directly dependent on the number of classes. The significance of classification is to use the least data to substitute the large amount original data with a basis of keeping the important feature information of defects. Take the steel shell specimen as an example, the resolution of collected image in the experiment is 359 281 and the number of sampling frames is 256. Therefore the original thermal image sequence includes 100,879 time sequences with a length of 256 which is a very large amount of data. But the amount of data can be significantly reduced after classification processing. If 100 classes were adopted, it only needs 100 sequences to substitute the original 100,879 sequences. The space compressing ratio of data compressing reaches 1008:1 and the compressing efficiency is very high with keeping the feature information of the original thermal images. Additionally, in the later fitting processing, it only needs to fit 100 sequences compared to the original 100,879 ones. This can save great processing time and significantly increase the fitting efficiency. Furthermore, there is to gain compressing benefit compared using other data fitting like TSR. For example, a typical thermal image sequence with 640 pixel 480 pixel 1000 frames can be classified into 150 classed. The global compressing algorithm has a time compressing ratio of 167:1, a space compressing ratio is 2048:1 and a global compressing ratio of 342,016:1. A thermal image sequence original data file is several hundred MB in its size may have a compressing file with only few KB. It is much smaller than that of TSR algorithm with a size over ten MB. For the time spent in thermal image compression computation, it is not applicable to compare iteration fitting algorithm to TSR fitting. Because TSR method computes directly the polynomial to fit a data sequence, while the algorithms presented in [10–12] are iteration fitting methods that need to go through 6 to dozens of times computation loops to do the same work. Although the speed of iteration fitting algorithm has been optimized, it is still dozens or hundred’s times slower than TSR fitting. This method proposed in this paper has significantly reduced the computation task by reducing 99.95–99.9% of the numbers of image sequences to be fitted, notwithstanding an increase of the K-means clustering calculation time. We used previous two sets of thermal image data to compare the computer processing time of the proposed method and that of TSR. The result is shown in Table 3.
The configuration of the computer: Intel Pentium P6200 CPU, dual cores, 2.13 GHz; 2G RAM; 320G hard disk; Windows XP OS; application programs developed in Delphi7. From Table 3, it can be seen that the proposed method offers faster speed than classic TSR method. Hereby we selected the data with largest class number for the comparison for which the compression effect can be considered the same for using both methods. 4. Conclusions All above, the study of K-mean algorithm based thermal wave image sequence global compressing algorithm can draw the conclusion as following. (1) For specimens made from different materials, provided a clear original thermal image can be obtained, this method can achieve an accurate classification of thermal image data. (2) This method not only keeps the original thermal image features, but also can classify and replace the data in a good order. For this experiment, the data space compressing ratio reached 1008:1 after processing and overall compressing ratio reached 40,000:1. This greatly saved storage space and consequential post processing computing workload. (3) In theory, the more classes, the closer the restored thermal image to original one. However, the experiment showed it a misunderstanding. It is necessary to set an appropriate number of classes to obtain optimum result. Normally, it is recommended to use 100–200 classes which could lead to optimum result.
Acknowledgments This work is supported by the National Natural Science Foundation of China (Grant Nos. 51275518 and 51305447) and the Natural Science Foundation of Shaanxi Province (Grant No. 2013JM7021). Appendix A In image data processing field, people are devoted to keep finding the way to assess the quality of an image after its processing. However, there is no uniform assessment standard up to date. The existing assessment methods are generally divided into two namely subjective ones and objective ones. A.1. Subjective assessment method For subjective assessment, an observer will judge on the quality of the visual effect of a testing image and mark it according to a set of pre-defined assessment criterion or his/her experience. Although this method allows the observer to make good use his/ her experience and reflects the intuitionistic quality well, it ignores quantitative measurement and not good for information processing due to lack of mathematical model description. And the result of assessment is limited by the observer’s knowledge background, emotion and degree of fatigue, etc. Subjective assessment method has limited application in real practice. A.2. Objective assessment method The advantage of objective assessment method is that it allows to perform quantitative calculation and measurement on the experimental images based on specific mathematical model to obtain information including luminance, contrast ratio, entropy, Root-Mean-Square error, peak value signal to noise ratio, spatial
25
J.-Y. Zhang et al. / Infrared Physics & Technology 64 (2014) 18–25
frequency, etc. In order to assess the quality of the processed image more accurately, this article compared four objective assessment parameters namely entropy, spatial frequency, Root-Mean-Square error and peak value signal to noise ratio. A brief introduction of the four parameters is as following: (1) Entropy is from information entropy theory. The more uncertain the variable is, the greater the value of entropy. Information entropy is an important parameter to measure the quantity of information in information theory. If a system was more unordered, the value of information entropy is greater. The definition of information entropy when it is applied to objective assessment of image is as following: N X H ¼ ðEi ðx; yÞlog2 Ei ðx; yÞÞ
ðA1Þ
i¼1
where E(x, y) is the occurring frequency of the grey level of any individual pixel point in the whole image, log 2Ei(x, y) is the corresponding logarithm of E(x, y). For any image, the greater the image’s information entropy is, the larger amount of information and the more details the image possesses. (2) Spatial frequency (SF) can represent the clearness level of an image. It is an important indication which is used to measure the level of information richness of the image. The higher the thermal wave image’s spatial frequency is, the richer the detail of the image is. The definition of spatial frequency is as following.
SF ¼
qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi RF2 þ CF2
ðA2Þ
When RF is row spatial frequency, CF is column spatial frequency
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u M X N u 1 X RF ¼ t ½Fðm; nÞ Fðm; n 1Þ2 MN m¼1 n¼2 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u N X M u 1 X CF ¼ t ½Fðm; nÞ Fðm 1; nÞ2 MN n¼1 m¼2
ðA3Þ
ðA4Þ
where F(m, n) represents the grey level of a pixel point located at (m, n) in coordinate. (3) Root-Mean-Square-Error (RMSE) is the error between processed image F and standard reference image R.
sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi PM PN 2 i¼1 j¼1 ðRði; jÞ Fði; jÞÞ RMSE ¼ MN
ðA5Þ
RMSE reflects the level of difference between processed image and standard reference image. Smaller value of RMSE shows that the change is small after applying enhancement algorithm. In order to facilitate the calculation, peak value signal to noise ratio which is
inverse proportional to Root-Mean-Square error can be taken as the calculation indication. (4) Peak value signal to noise ratio (PSNR) is defined as following:
PSNR ¼ 10 log
255 255 RMSE2
ðA6Þ
The higher the value of PSNR is, the better the processing effect of the image is.
References [1] M. Genest, M. Martinez, N. Mrad, Pulsed thermography for non-destructive evaluation and damage growth monitoring of bonded repairs, Compos. Struct. 88 (1) (2009) 112–120. [2] M.R. Valluzzi, E. Grinzato, C. Pellegrino, IR thermography for interface analysis of FRP laminates externally bonded to RC beams, Mater. Struct. 42 (1) (2009) 25–34. [3] V. Crupi, G. Chiofalo, E. Guglielmino, Using infrared thermography in low cycle fatigue studies of welded joints, Welding J. 89 (9) (2010) 195–200. [4] R. Jones, M. Krishnapillai, K. Cairns, Application of infrared thermography to study crack growth and fatigue life extension procedures, Fatigue Fract. Eng. Mater. Struct. 33 (12) (2010) 871–884. [5] T. Hannes, G. Stefan, Pulsed thermography of CFC monoblock divertor components, Fusion Eng. Des. 84 (11) (2009) 1867–1870. [6] X.L. Yang, T. Jang, L.C. Feng, Thermographic testing for impact damage of airplane composite, Nondestruct. Test. 31 (2) (2009) 120–122. [7] Y.Y. Zhao, Y.P. Tian, Data fitting method in pulsed phase thermography, Nondestruct. Test. 29 (11) (2007) 637–640. [8] X.W. Guo, V. Vavilov, V. Shiryaev, Thermal nondestructive testing of corrosion in aviation aluminum panels and data processing algorithms, Chinese J. Mech. Eng. 45 (3) (2009) 208–213. [9] S.M. Shepard, T. Ahmed, B.A. Rubadeux, D. Wang, J.R. Lhota, Synthetic processing of pulsed thermographic data for inspection of turbine components, Insight Brit. Inst. NDT 43 (9) (2001) 587–589. [10] Y. Zhang, J.Y. Zhang, J.X. Huang, Infrared thermal imaging data fitting method based on theoretical model of infrared thermal detection, Infrared 33 (4) (2012) 38–41. [11] Y. Zhang, J.Y. Zhang, X.R. Huang, J.X. Huang, The infrared thermal imaging data fitting method based on genetic algorithm, Nondestruct. Test. 34 (10) (2012) 38–41. [12] H. Chen, S.G. Deng, Y.R. Fan, Application of homotopy alternative iteration method in double exponential fitting, Comput. Eng. Appl. 43 (25) (2007) 204– 205. [13] Z. Lin, Y. Xing, Survey of feature extraction approaches for time series classification, Comput. Syst. Appl. 21 (10) (2012) 224–229. [14] L.L. Li, A review on classification algorithms in data mining, J. Chongqing Normal Univ. (Nat. Sci.) 28 (4) (2011) 44–47. [15] W. Zhen, M.Q. Tong, The biological and physical mechanism of infrared thermal imaging in medical diagnosis, Life Sci. Instr. 3 (2) (2005) 20–22. [16] H. Jiang, R. Zhang, Key technologies in registration and fusion for infrared and visible images, Infrared Laser Eng. 35 (10) (2006) 7–12. [17] W. Zhao, M. Zhang, W.H. Li, Improved K-means clustering algorithm on space, Appl. Res. Comp. 25 (7) (2008) 1995–1997. [18] T. Huang, S.H. Liu, Y.N. Tan, Research of clustering algorithm based on Kmeans, Comp. Technol. Dev. 21 (7) (2011) 54–57. [19] Shokri Z. Selem, M.A. Imail, K-means-type algorithms: a generalized convergence theorem and characterization of local optimality, IEEE Trans. Pattern Anal. Mach. Int. 6 (1) (1984) 5–50.