- Email: [email protected]
TFT-LCD mura defect detection using DCT and the dual-γ piecewise exponential transform
Precision Engineering xxx (xxxx) xxx–xxx
Contents lists available at ScienceDirect
Precision Engineering journal homepage: www.elsevier.com/locate/precision
TFT-LCD mura defect detection using DCT and the dual-γ piecewise exponential transform Shiqun Jina, Chao Jia,∗, Chengchen Yanb, Jinyu Xingb a
Key Laboratory of Special Display Technology of the Ministry of Education, National Engineering Laboratory of Special Display Technology, National Key Laboratory of Advanced Display Technology, Academy of Photoelectric Technology, Hefei University of Technology, Hefei, 230009, China b School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology, Hefei, 230009, China
A R T I C LE I N FO
A B S T R A C T
Keywords: TFT-LCD Mura defect Discrete cosine transform Piecewise exponential transform
This paper proposes a mura defect detection method that is based on the discrete cosine transform and the dual-γ piecewise exponential transform for thin-film transistor liquid crystal display panels. First, the background of the original image with mura defects is reconstructed by means of the discrete cosine transform, and the mura image is obtained by subtracting the reconstructed image from the original image. Second, the dual-γ piecewise exponential transform method is proposed for suppressing residual background information and improving the contrast of the image. Finally, Otsu's method is adopted to segment the muras completely. The experimental results suggest that the proposed method effectively increases the low contrast of muras by at least 14 times compared to the traditional discrete cosine transform background reconstruction method and at least 2 times compared to the polynomial fitting method. In addition, the method improves the accuracy of mura defect identification, and the detection effect is stable for various non-uniform backgrounds. These results demonstrate that the proposed method has high accuracy and robustness.
1. Introduction With the widespread use of thin-film transistor liquid crystal display (TFT-LCD) panels for mobile phones, tablets and computers, display panel defect detection has been considered by an increasing number of TFT-LCD manufacturers. Mura defects in TFT-LCD panel images are characterized by low contrast, blurry contours and irregular shape patterns [1], which are difficult to detect. The main mura defect detection methods that are adopted by most scholars include the feature extraction method [2–5] and the background suppression method [6–12]. The feature extraction method is a fast and efficient method that directly extracts the features of defects. However, the method shows a relatively low degree of flexibility because it requires features of defects to be pre-set. The background suppression method, which is based on background reconstruction and difference, is effective in segmenting mura defects from the original background, which remains one of the main research interests in current studies. Tsai et al. attempted to solve the problem of uneven brightness of LCD backgrounds by employing the one-dimensional Fourier transform, Gabor filter and singular value decomposition (SVD) [6]. However, the mura defects with low contrast and blurry contours were likely to be left out, thereby resulting in decreased detection
∗
accuracy. Lee et al. put forward a regression and fitting-based background reconstruction algorithm for defect detection that compares the test image and the background reconstruction image [1,7]. Due to the complex regression and fitting processes, the algorithm showed low efficiency and poor real-time performance. Bi Xin employed the Gabor wavelet transform to filter out textured backgrounds from TFT-LCD panels [8]. However, the experiment failed to offer effective solutions to the problem of uneven brightness of backgrounds. Zhang Yu et al. introduced the method that detects mura defects by removing backgrounds with the least-square and polynomial curve fitting methods. At the same time, a rule-based fuzzy classifier was developed for defect identification [9]. As for the limitations of the method, the classifier required a massive number of feature parameters for training; furthermore, as the order of the polynomial rose, Runge's phenomenon became more likely to occur in the fitting backgrounds. Hence, it is not suitable for real-time image processing. Li Kun employed the bicubic Bspline surface fitting method to eliminate Runge's phenomenon in polynomial fitting and added a fairing item to improve the background suppression [10]. However, the fitting backgrounds that were processed by the least-square method were too similar to the original images, which diminished the mura defects in the difference images. Chen et al. put forward a discrete cosine transform (DCT)-based background
Corresponding author. E-mail address: [email protected] (C. Ji).
https://doi.org/10.1016/j.precisioneng.2018.07.006 Received 1 August 2017; Received in revised form 14 July 2018; Accepted 17 July 2018 0141-6359/ © 2018 Elsevier Inc. All rights reserved.
Please cite this article as: Jin, S., Precision Engineering (2018), https://doi.org/10.1016/j.precisioneng.2018.07.006
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
First, an original image was transformed to its frequency domain by DCT and all coefficients were reset, except the DC coefficients that represent the low-frequency (LF) signals of the image. Second, background reconstruction was carried out by means of inverse discrete cosine transform (IDCT). Third, most of the background information was filtered out by subtracting the reconstructed image from the original image to obtain the mura image. Fourth, to further enhance the contrast of the mura defect area, Otsu's method was employed to obtain a threshold value, based on which the γ-exponential transform of the background and mura defect area was performed respectively. Fifth, image segmentation was carried out with Otsu's method. Lastly, three evaluation methods were introduced in this paper for quantitative evaluation of the detection results, which further demonstrated the high accuracy, efficiency and robustness of the proposed method.
2. DCT-based background reconstruction Discrete Cosine Transform (DCT) is often employed to compress signals and images in signal processing and image processing. It is used because of the strong “energy concentration” characteristic of DCT; most of the energy of natural signals (including sounds and images) is concentrated in the low-frequency part after DCT. Based on this principle, discrete cosine transform is adopted for background reconstruction. For an M × N image, f(m,n) represents the gray level function of the image. After the DCT process, a DCT coefficient matrix F(u,v) is obtained, where the coefficients, from the upper-left to the lower-right corners, represent even distribution (low-frequency), minor variation (medium-frequency) and dramatic variation (high-frequency) in the gray level. Since an even variation of the background gray level was observed in the LCD, it corresponded to the upper-left corner of the frequency domain. After the DCT process, the coefficients that corresponded to the first row (u = 0) and the first column (v = 0) were retained while the remaining coefficients were reset. Then, a reconstructed background image was produced through IDCT. As shown in Fig. 1, to verify the effect of the described theory on separating mura defects from the backgrounds, an image with a mura defect was selected for carrying out background reconstruction and producing a difference image. According to Fig. 1, it is clear that the DCT is effective in background reconstruction. However, the mura image has “crossed veins” in both the horizontal and vertical directions that affect the quality of reconstruction and the follow-on segmentation of the mura and background images. According to DCT theory [12], the “crossed veins” appear in the image after subtraction because the energy that is generated by the mura defect area is transmitted along the veins of its fundamental frequency after being processed by the DCT. To suppress these “crossed veins”, the dual-γ piecewise exponential transform based on Otsu's method was introduced for contrast enhancement in this paper.
Fig. 1. DCT-based background reconstruction images. (a) Original mura image; (b) Background reconstruction image; (c) Difference image; and (d) Net image of the difference image.
Fig. 2. Dual-γ piecewise exponential transformation. The horizontal axis indicates the gray-level values before transformation. The vertical axis indicates the gray-level values after transformation. The segmentation point T represents the threshold that was obtained by Otsu's method.
3. Contrast enhancement with dual-γ piecewise exponential transform based on Otsu's method Otsu's method is a widely used thresholding method of image segmentation. Based on the gray-level histogram of an image, the method selects the gray-level value that corresponds to the maximum inter-class variance as the optimum threshold that separates the two classes of the image [13]. It is an adaptive thresholding method. Determining the threshold with the maximum inter-class variance is a convenient and efficient automatic thresholding method that does not require manual parameter setting. The threshold represents the segmentation point for gray-level transformation of the two sections of the gray-level line with the γ-exponential function. In other words, the gray-level value in [0, T] is used as γ1, while that in [T+1, L-1] serves as γ2 for the exponential transformation. The expression is
reconstruction technology that utilized the data compression feature of DCT to subtract the reconstructed background from the original image for defect separation [11,12]. Despite its simple calculation and impressive real-time performance, the method was inefficient in background suppression under complex illumination and uneven defect backgrounds, thereby leading to residues of partial backgrounds in the mura images after subtraction. In this study, an in-depth analysis was conducted on the aforementioned DCT-based background reconstruction method [11] that was put forward by Chen. A mura defect detection method that is based on DCT and the dual-γ piecewise exponential transform was introduced. 2
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Fig. 3. Defect detection results that were obtained by the proposed method with various values of γ . The range of γ is from 1 to 2. C denotes the corresponding defectbackground contrast after dual-γ piecewise exponential transformation. The detailed areas of these images are enlarged and marked in yellow. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
Fig. 5. Experimental process.
The average defect-background contrast C is introduced for evaluating contrast enhancement effects. C can be formulated as
C = ( LM − LB )/ LB
(2)
Fig. 4. Mura image before and after dual-γ piecewise exponential transformation. (a) Mura image before transformation. (b) Net image before transformation. (c) Mura image after transformation. (d) Net image after transformation.
where LM and LB express the average gray value of the defect area and the background after processing with the algorithm, respectively. The detailed steps for obtaining C are as follows:
(ki) γ1, 0 ≤ i ≤ T j=⎧ γ1 γ2 ⎨ ( ⎩ kT ) + [k (i − T )] , T + 1 ≤ i ≤ L − 1
(1) In the last step of our algorithm, the threshold T of image segmentation is calculated; (2) Based on this threshold T, the image that has been enhanced by the proposed algorithm is classified into the defect area and the background area. (3) The average gray value of the defect area LM and the average gray value of the background area LB are calculated; (4) C is calculated according to Eq. (2) in the text.
(1)
where j represents the new gray-level value that is transformed from the gray-level value i in the original image. To reduce the need for parameter setting, we assume that γ2 = 1/γ1 = γ and k is an appropriate positive integer (with a default value of 1). The correlation between the gray-level values before and after transformation is as shown in Fig. 2. 3
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Fig. 6. Mura Images and experimental results. (a)∼(c) spot, line and region muras; (d)∼(f) detection results of the DCT method; (g)∼(i) detection results of the polynomial fitting method; (j)∼(l) detection results of the proposed method.
contour of the target area in the image have been enhanced while the “crossed veins” in the background have been suppressed. Finally, Otsu's method was employed to directly separate the mura defect from the image in a fast and efficient manner.
To determine the best values of the gamma parameters, we selected a 250 × 250-pixel-resolution sample for a group of experiment. The range of γ discussed in this paper is 1 ≤ γ≤ 2 (γ = 1 indicates no transform). The experimental results are shown in Fig. 3. Tiny background residuals are marked in yellow in the figure. Taking into account both the contrast and the detection effect, we determine that the optimal value range of γ is 1.2–1.8. The image before and after the difference process is shown in Fig. 4. After the dual-γ piecewise exponential transform, the contrast and 4
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Table 1 Evaluation and comparison of average contrast C. LM and LB denote the average gray value of the defect area and the background after processing with the algorithm, respectively. Method
Shape
Polynomial fitting
DCT
Spot Line Region
Our Method
LM
LB
C
LM
LB
C
LM
LB
C
18.26 10.92 10.82
1.19 1.88 1.87
14.38 4.81 4.79
18.00 10.69 10.56
0.53 0.37 0.41
33.03 27.76 24.45
31.75 21.96 20.92
0.06 0.32 0.22
563.85 68.50 96.08
Fig. 7. Ten artificial mura images. (a)∼(j) The differences of gray value between the mura and background range from 1 to 10.
4. Experiment and the analysis of the results
Table 2 Comparison of ME among various methods. The differences of gray value between the mura and background range from 1 to 10. Gray value difference
10 9 8 7 6 5 4 3 2 1
4.1. Experimental results
Misclassification error Polynomial fitting
DCT method
Our method
0.0035 0.0033 0.0036 0.0033 0.0034 0.0036 0.0037 0.0025 0.0032 0.0247
0.0027 0.0021 0.0020 0.0089 0.0081 0.0236 0.0279 0.0365 0.0778 0.9287
0.0023 0.0021 0.0012 0.0023 0.0011 0.0011 0.0013 0.0013 0.0017 0.0041
To evaluate the effectiveness of the algorithm that is proposed in this paper, three types of sample images of 250 × 250-pixel resolution with spot-mura, line-mura and region-mura defects were selected. The experimental process is as shown in Fig. 5. Mura images and the experimental results are shown in Fig. 6. According to the experimental results, the mura images that were segmented using the DCT method contain many background residues, the mura images that were segmented using the polynomial fitting method have fewer residues and our method has a better detection effect compared to the other two methods.
4.2. Evaluation of the experimental results Table 3 Comparison of computation time among various methods. No. of image
20 20 20 20 20 20
Image size (pixel)
250 × 250 500 × 500 750 × 750 1000 × 1000 2000 × 2000 3000 × 3000
To evaluate the effects of the methods in mura defect detection, the concepts of enhancement [14], misclassification error [13] and computation time were introduced.
Average computation time (ms) DCT
Polynomial fitting
Our method
28 95 226 392 1565 3996
65 280 618 1107 4527 11084
41 111 283 485 1962 4394
4.2.1. Enhancement Enhancement refers to the condition that the mura defect becomes increasingly significant and visible after background suppression. It can be evaluated based on the given average defect-background contrast C. The detailed steps for obtaining C utilize Eq. (2). Comparisons of C among the DCT method, the polynomial fitting method and our method are shown in Table 1. R is defined for calculating the ratio between two methods. R can be formulated as 5
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Fig. 8. The average computation time of the three methods. The horizontal axis indicates the image size. The vertical axis indicates the computation time. The line graphs of the three methods are respectively distinguished by three line types as shown in the figure. Table 4 Robustness assessment under various background conditions. Five samples were tested in the experiment, among which the difference between the highest and lowest gray values of the background increased from 74 to 132. Minimal gray value of background
Maximal gray value of background
Gray value difference of background
109
183
74
1099
103
192
89
1062
96
200
104
1071
87
206
119
1078
80
212
132
1024
6
Mura image
Binary image
Area (pixel)
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Table 5 Robustness assessment under various image sizes. Three sizes of samples were tested in the experiment, among which the background conditions are the same as Table 4. Image size (pixel)
250 × 250 500 × 500 750 × 750
Mura area (pixel) Sample 1
Sample 2
Sample 3
Sample 4
Sample 5
1099 4252 10028
1062 4296 9551
1071 4327 9698
1078 4319 9229
1024 4131 9746
(3)
R = C1/ C2
27.5445 71.8971 261.3217
0.04% 0.03% 0.04%
During the inspection process, the size of the image and the lighting conditions may affect the detection results. Two sets of experiments were performed to evaluate the robustness of the proposed method. In order to verify the impact of the lighting environment on the experimental results, we set up the first set of experiments. Five samples of 250 × 250 pixels were tested in the experiment, among which the difference between the highest and lowest gray values of the background increased from 74 to 132. The detection results are shown in Table 4 and we calculated the area of the mura segmented from the background. The standard deviation of these data is 27.54 pixels according to Eq. (5) and relative standard deviation (RSD) is 0.04% according to Eq. (6), which demonstrates that the difference in detection results is tiny in different lighting environments. In order to verify the impact of the image size on the experimental results, we set up a second set of experiments. The images of 500 × 500 pixels and 750 × 750 pixels were tested in the same environment as the first set of experiments. The detection results are shown in Table 5 and we can find that the relative standard deviation of the test results is stable at 0.03%–0.04%. According to the above experiment, we can draw this conclusion that the proposed method has high robustness.
4.2.2. Misclassification error To evaluate the accuracy of the proposed method, ten gray-value levels of artificial mura defects were added to an LCD panel that had even gray distribution of background, in which the differences in gray values ranged from 1 to 10, as shown in Fig. 7. The misclassification error (ME) is introduced for comparison of the accuracy [12]. ME can be formulated as
BC ∩ BS + MC ∩ MS BC + MC
relative standard deviation
4.3. Robustness assessment
C1 and C2 denote the contrasts of the two methods. R varies with the sample of mura shape. We take the minimum value as the improvement ratio between the methods. The method that is proposed in this paper effectively improved the mura defect-background contrast by at least 14 times compared to the conventional DCT method and at least 2 times compared to the polynomial fitting method.
ME = 1 −
standard deviation
(4)
where BC and MC denote the correct background and mura of the artificial image, respectively, and BS and MS denote the background and mura pixels in the segmented image, respectively. The values of ME are listed in Table 2. The lower the value of ME is, the higher the accuracy of the detection is. From Table 2, we conclude that the polynomial fitting method has a good detection effect and high accuracy. In the case of a large gray difference between the background and the mura, the DCT method performs well. However, with the decrease of the gray difference, the detection effectiveness worsens and the accuracy decreases. The proposed method outperforms the other methods.
n
σ=
∑i = 1 Δsi2 n−1
RSD = σ / s
(5) (6)
where Δsi = si − s , si denotes the area of the mura in each background and s denotes the average area of all muras. These areas are in units of pixel. The number of test images is n.
5. Conclusions
4.2.3. Computation time It is significant to compare the proposed method, the DCT method and the polynomial fitting method in terms of efficiency. The value of ME and the computation time are employed to evaluate the efficiency of the algorithms. The smaller the value of ME and the shorter the run time, the higher the efficiency. ME is calculated in Table 2 and the computation times of the three methods are listed in Table 3. Six groups of experiments were performed on a personal computer with an Intel CPU of 2.5 GHz and 8 GB RAM. The image sizes of these six groups of experiments were 250 × 250 pixels, 500 × 500 pixels, 750 × 750 pixels, 1000 × 1000 pixels, 2000 × 2000 pixels, 3000 × 3000 pixels respectively. In each group of experiments, a set of 20 images were tested to calculate the average computation time. The DCT method, the proposed method and the polynomial fitting method respectively take 28 ms, 41 ms and 65 ms to complete the detection for dealing with the test images of 250 × 250 pixels. The average computation time of the three methods are shown in Fig. 8 and the proposed method is between the DCT method and the polynomial fitting method. As the size of the image increases, the computation time of all algorithms increases. However, the computation time of the polynomial fitting method has the fastest growth rate and is almost 2.5 times as our method in processing images of 3000 × 3000 pixels.
This paper proposed a mura defect detection method that is based on DCT and the dual-γ piecewise exponential transform. Aiming at improving the mura defect detection rates and detection results, the main components of a background are extracted in the frequency domain for reconstruction. Since the background cannot be completely filtered out, Otsu's method is adopted to determine the threshold as the segmentation point, while the mura defect area is enhanced by means of the dual-γ piecewise exponential transform. Lastly, Otsu's method is used for defect detection through image segmentation. In addition, to compare the mura defect detection results of the DCT method, the polynomial fitting method and the proposed method, three indices, namely, enhancement, misclassification error, and computation time, are introduced for quantitative evaluation. Compared to the traditional DCT background reconstruction method and the polynomial fitting method, the proposed method increases the contrast of the mura defect area by at least 14 times and 2 times, respectively. In addition, the method improves the accuracy of mura defect identification and can complete the detection of a 250 × 250-pixel image efficiently in 41 ms. Moreover, the robustness assessment experiments demonstrate that the proposed method has high robustness. 7
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
[6] Tsai DM, Hung CY. Automatic defect inspection of patterned thin film transistorliquid crystal display (TFT-LCD) panels using one-dimensional Fourier reconstruction and wavelet decomposition. Int J Prod Res 2005;43(21):4589–607. [7] Chen SL, Chang JH. TFT-LCD Mura defects automatic inspection system using linear regression diagnostic model. Proc Inst Mech Eng Part B J Eng Manufact 2008;222(11):1489–501. [8] Bi Xin. Machine vision inspection method of Mura defect for TFT-LCD. J Mech Eng 2010;46(12):13–9. [9] Zhang Yu, Zhang Jian. Automatic blemish inspection for TFT-LCD based on polynomial surface fitting. Opto-Electronic Eng 2006;33(10):108–14. [10] Li Kun, Li Hui, et al. Background suppression of LCD Mura defect using B-spline surface fitting. Opto-Electronic Eng 2014;41(2):33–9. [11] Chen LC, Kuo CC. Automatic TFT-LCD mura defect inspection using discrete cosine transform-based background filtering and 'just noticeable difference' quantification strategies. Meas Sci Technol 2008;19(1):015507. [12] Chen LC, Chien CH, Nguyen XL. An effective image segmentation method for noisy low-contrast unbalanced background in Mura defects using balanced discrete-cosine-transfer (BDCT). Precis Eng 2013;37(2):336–44. [13] Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 1979;9(1):62–6. [14] Xie Rui, Li Gang. High quality background modeling of LCD-Mura defect. J Comput Appl 2016(04):1151–5.
Acknowledgement This work was supported by Ministry of Science and Technology of the People's Republic of China (Grant No.2013YQ220749). References [1] Lee JY, Yoo SI. Automatic detection of region-mura defect in TFT-LCD. IEICE Trans Info Syst 2004;87(10):2371–8. [2] Tseng DC, Lee YC, Shie CE. LCD mura detection with multi-image accumulation and multi-resolution background subtraction. Int J Innov Comput Inf Contr 2012;8(7):4837–50. [3] Oh JH, Kwak DM, Lee KB, et al. Line defect detection in TFT-LCD using directional filter bank and adaptive multilevel thresholding. Key Eng Mater 2004;270–273:233–8. [4] Taniguchi K, Ueta K, Tatsumi S. A mura detection method. Pattern Recogn 2006;39(6):1044–52. [5] Song YC, Choi DH, Park KH. Morphological blob-mura defect detection method for TFT-LCD panel inspection.Knowledge-based intelligent information and engineering systems. International conference, Kes. 2004.
8
Contents lists available at ScienceDirect
Precision Engineering journal homepage: www.elsevier.com/locate/precision
TFT-LCD mura defect detection using DCT and the dual-γ piecewise exponential transform Shiqun Jina, Chao Jia,∗, Chengchen Yanb, Jinyu Xingb a
Key Laboratory of Special Display Technology of the Ministry of Education, National Engineering Laboratory of Special Display Technology, National Key Laboratory of Advanced Display Technology, Academy of Photoelectric Technology, Hefei University of Technology, Hefei, 230009, China b School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology, Hefei, 230009, China
A R T I C LE I N FO
A B S T R A C T
Keywords: TFT-LCD Mura defect Discrete cosine transform Piecewise exponential transform
This paper proposes a mura defect detection method that is based on the discrete cosine transform and the dual-γ piecewise exponential transform for thin-film transistor liquid crystal display panels. First, the background of the original image with mura defects is reconstructed by means of the discrete cosine transform, and the mura image is obtained by subtracting the reconstructed image from the original image. Second, the dual-γ piecewise exponential transform method is proposed for suppressing residual background information and improving the contrast of the image. Finally, Otsu's method is adopted to segment the muras completely. The experimental results suggest that the proposed method effectively increases the low contrast of muras by at least 14 times compared to the traditional discrete cosine transform background reconstruction method and at least 2 times compared to the polynomial fitting method. In addition, the method improves the accuracy of mura defect identification, and the detection effect is stable for various non-uniform backgrounds. These results demonstrate that the proposed method has high accuracy and robustness.
1. Introduction With the widespread use of thin-film transistor liquid crystal display (TFT-LCD) panels for mobile phones, tablets and computers, display panel defect detection has been considered by an increasing number of TFT-LCD manufacturers. Mura defects in TFT-LCD panel images are characterized by low contrast, blurry contours and irregular shape patterns [1], which are difficult to detect. The main mura defect detection methods that are adopted by most scholars include the feature extraction method [2–5] and the background suppression method [6–12]. The feature extraction method is a fast and efficient method that directly extracts the features of defects. However, the method shows a relatively low degree of flexibility because it requires features of defects to be pre-set. The background suppression method, which is based on background reconstruction and difference, is effective in segmenting mura defects from the original background, which remains one of the main research interests in current studies. Tsai et al. attempted to solve the problem of uneven brightness of LCD backgrounds by employing the one-dimensional Fourier transform, Gabor filter and singular value decomposition (SVD) [6]. However, the mura defects with low contrast and blurry contours were likely to be left out, thereby resulting in decreased detection
∗
accuracy. Lee et al. put forward a regression and fitting-based background reconstruction algorithm for defect detection that compares the test image and the background reconstruction image [1,7]. Due to the complex regression and fitting processes, the algorithm showed low efficiency and poor real-time performance. Bi Xin employed the Gabor wavelet transform to filter out textured backgrounds from TFT-LCD panels [8]. However, the experiment failed to offer effective solutions to the problem of uneven brightness of backgrounds. Zhang Yu et al. introduced the method that detects mura defects by removing backgrounds with the least-square and polynomial curve fitting methods. At the same time, a rule-based fuzzy classifier was developed for defect identification [9]. As for the limitations of the method, the classifier required a massive number of feature parameters for training; furthermore, as the order of the polynomial rose, Runge's phenomenon became more likely to occur in the fitting backgrounds. Hence, it is not suitable for real-time image processing. Li Kun employed the bicubic Bspline surface fitting method to eliminate Runge's phenomenon in polynomial fitting and added a fairing item to improve the background suppression [10]. However, the fitting backgrounds that were processed by the least-square method were too similar to the original images, which diminished the mura defects in the difference images. Chen et al. put forward a discrete cosine transform (DCT)-based background
Corresponding author. E-mail address: [email protected] (C. Ji).
https://doi.org/10.1016/j.precisioneng.2018.07.006 Received 1 August 2017; Received in revised form 14 July 2018; Accepted 17 July 2018 0141-6359/ © 2018 Elsevier Inc. All rights reserved.
Please cite this article as: Jin, S., Precision Engineering (2018), https://doi.org/10.1016/j.precisioneng.2018.07.006
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
First, an original image was transformed to its frequency domain by DCT and all coefficients were reset, except the DC coefficients that represent the low-frequency (LF) signals of the image. Second, background reconstruction was carried out by means of inverse discrete cosine transform (IDCT). Third, most of the background information was filtered out by subtracting the reconstructed image from the original image to obtain the mura image. Fourth, to further enhance the contrast of the mura defect area, Otsu's method was employed to obtain a threshold value, based on which the γ-exponential transform of the background and mura defect area was performed respectively. Fifth, image segmentation was carried out with Otsu's method. Lastly, three evaluation methods were introduced in this paper for quantitative evaluation of the detection results, which further demonstrated the high accuracy, efficiency and robustness of the proposed method.
2. DCT-based background reconstruction Discrete Cosine Transform (DCT) is often employed to compress signals and images in signal processing and image processing. It is used because of the strong “energy concentration” characteristic of DCT; most of the energy of natural signals (including sounds and images) is concentrated in the low-frequency part after DCT. Based on this principle, discrete cosine transform is adopted for background reconstruction. For an M × N image, f(m,n) represents the gray level function of the image. After the DCT process, a DCT coefficient matrix F(u,v) is obtained, where the coefficients, from the upper-left to the lower-right corners, represent even distribution (low-frequency), minor variation (medium-frequency) and dramatic variation (high-frequency) in the gray level. Since an even variation of the background gray level was observed in the LCD, it corresponded to the upper-left corner of the frequency domain. After the DCT process, the coefficients that corresponded to the first row (u = 0) and the first column (v = 0) were retained while the remaining coefficients were reset. Then, a reconstructed background image was produced through IDCT. As shown in Fig. 1, to verify the effect of the described theory on separating mura defects from the backgrounds, an image with a mura defect was selected for carrying out background reconstruction and producing a difference image. According to Fig. 1, it is clear that the DCT is effective in background reconstruction. However, the mura image has “crossed veins” in both the horizontal and vertical directions that affect the quality of reconstruction and the follow-on segmentation of the mura and background images. According to DCT theory [12], the “crossed veins” appear in the image after subtraction because the energy that is generated by the mura defect area is transmitted along the veins of its fundamental frequency after being processed by the DCT. To suppress these “crossed veins”, the dual-γ piecewise exponential transform based on Otsu's method was introduced for contrast enhancement in this paper.
Fig. 1. DCT-based background reconstruction images. (a) Original mura image; (b) Background reconstruction image; (c) Difference image; and (d) Net image of the difference image.
Fig. 2. Dual-γ piecewise exponential transformation. The horizontal axis indicates the gray-level values before transformation. The vertical axis indicates the gray-level values after transformation. The segmentation point T represents the threshold that was obtained by Otsu's method.
3. Contrast enhancement with dual-γ piecewise exponential transform based on Otsu's method Otsu's method is a widely used thresholding method of image segmentation. Based on the gray-level histogram of an image, the method selects the gray-level value that corresponds to the maximum inter-class variance as the optimum threshold that separates the two classes of the image [13]. It is an adaptive thresholding method. Determining the threshold with the maximum inter-class variance is a convenient and efficient automatic thresholding method that does not require manual parameter setting. The threshold represents the segmentation point for gray-level transformation of the two sections of the gray-level line with the γ-exponential function. In other words, the gray-level value in [0, T] is used as γ1, while that in [T+1, L-1] serves as γ2 for the exponential transformation. The expression is
reconstruction technology that utilized the data compression feature of DCT to subtract the reconstructed background from the original image for defect separation [11,12]. Despite its simple calculation and impressive real-time performance, the method was inefficient in background suppression under complex illumination and uneven defect backgrounds, thereby leading to residues of partial backgrounds in the mura images after subtraction. In this study, an in-depth analysis was conducted on the aforementioned DCT-based background reconstruction method [11] that was put forward by Chen. A mura defect detection method that is based on DCT and the dual-γ piecewise exponential transform was introduced. 2
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Fig. 3. Defect detection results that were obtained by the proposed method with various values of γ . The range of γ is from 1 to 2. C denotes the corresponding defectbackground contrast after dual-γ piecewise exponential transformation. The detailed areas of these images are enlarged and marked in yellow. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
Fig. 5. Experimental process.
The average defect-background contrast C is introduced for evaluating contrast enhancement effects. C can be formulated as
C = ( LM − LB )/ LB
(2)
Fig. 4. Mura image before and after dual-γ piecewise exponential transformation. (a) Mura image before transformation. (b) Net image before transformation. (c) Mura image after transformation. (d) Net image after transformation.
where LM and LB express the average gray value of the defect area and the background after processing with the algorithm, respectively. The detailed steps for obtaining C are as follows:
(ki) γ1, 0 ≤ i ≤ T j=⎧ γ1 γ2 ⎨ ( ⎩ kT ) + [k (i − T )] , T + 1 ≤ i ≤ L − 1
(1) In the last step of our algorithm, the threshold T of image segmentation is calculated; (2) Based on this threshold T, the image that has been enhanced by the proposed algorithm is classified into the defect area and the background area. (3) The average gray value of the defect area LM and the average gray value of the background area LB are calculated; (4) C is calculated according to Eq. (2) in the text.
(1)
where j represents the new gray-level value that is transformed from the gray-level value i in the original image. To reduce the need for parameter setting, we assume that γ2 = 1/γ1 = γ and k is an appropriate positive integer (with a default value of 1). The correlation between the gray-level values before and after transformation is as shown in Fig. 2. 3
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Fig. 6. Mura Images and experimental results. (a)∼(c) spot, line and region muras; (d)∼(f) detection results of the DCT method; (g)∼(i) detection results of the polynomial fitting method; (j)∼(l) detection results of the proposed method.
contour of the target area in the image have been enhanced while the “crossed veins” in the background have been suppressed. Finally, Otsu's method was employed to directly separate the mura defect from the image in a fast and efficient manner.
To determine the best values of the gamma parameters, we selected a 250 × 250-pixel-resolution sample for a group of experiment. The range of γ discussed in this paper is 1 ≤ γ≤ 2 (γ = 1 indicates no transform). The experimental results are shown in Fig. 3. Tiny background residuals are marked in yellow in the figure. Taking into account both the contrast and the detection effect, we determine that the optimal value range of γ is 1.2–1.8. The image before and after the difference process is shown in Fig. 4. After the dual-γ piecewise exponential transform, the contrast and 4
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Table 1 Evaluation and comparison of average contrast C. LM and LB denote the average gray value of the defect area and the background after processing with the algorithm, respectively. Method
Shape
Polynomial fitting
DCT
Spot Line Region
Our Method
LM
LB
C
LM
LB
C
LM
LB
C
18.26 10.92 10.82
1.19 1.88 1.87
14.38 4.81 4.79
18.00 10.69 10.56
0.53 0.37 0.41
33.03 27.76 24.45
31.75 21.96 20.92
0.06 0.32 0.22
563.85 68.50 96.08
Fig. 7. Ten artificial mura images. (a)∼(j) The differences of gray value between the mura and background range from 1 to 10.
4. Experiment and the analysis of the results
Table 2 Comparison of ME among various methods. The differences of gray value between the mura and background range from 1 to 10. Gray value difference
10 9 8 7 6 5 4 3 2 1
4.1. Experimental results
Misclassification error Polynomial fitting
DCT method
Our method
0.0035 0.0033 0.0036 0.0033 0.0034 0.0036 0.0037 0.0025 0.0032 0.0247
0.0027 0.0021 0.0020 0.0089 0.0081 0.0236 0.0279 0.0365 0.0778 0.9287
0.0023 0.0021 0.0012 0.0023 0.0011 0.0011 0.0013 0.0013 0.0017 0.0041
To evaluate the effectiveness of the algorithm that is proposed in this paper, three types of sample images of 250 × 250-pixel resolution with spot-mura, line-mura and region-mura defects were selected. The experimental process is as shown in Fig. 5. Mura images and the experimental results are shown in Fig. 6. According to the experimental results, the mura images that were segmented using the DCT method contain many background residues, the mura images that were segmented using the polynomial fitting method have fewer residues and our method has a better detection effect compared to the other two methods.
4.2. Evaluation of the experimental results Table 3 Comparison of computation time among various methods. No. of image
20 20 20 20 20 20
Image size (pixel)
250 × 250 500 × 500 750 × 750 1000 × 1000 2000 × 2000 3000 × 3000
To evaluate the effects of the methods in mura defect detection, the concepts of enhancement [14], misclassification error [13] and computation time were introduced.
Average computation time (ms) DCT
Polynomial fitting
Our method
28 95 226 392 1565 3996
65 280 618 1107 4527 11084
41 111 283 485 1962 4394
4.2.1. Enhancement Enhancement refers to the condition that the mura defect becomes increasingly significant and visible after background suppression. It can be evaluated based on the given average defect-background contrast C. The detailed steps for obtaining C utilize Eq. (2). Comparisons of C among the DCT method, the polynomial fitting method and our method are shown in Table 1. R is defined for calculating the ratio between two methods. R can be formulated as 5
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Fig. 8. The average computation time of the three methods. The horizontal axis indicates the image size. The vertical axis indicates the computation time. The line graphs of the three methods are respectively distinguished by three line types as shown in the figure. Table 4 Robustness assessment under various background conditions. Five samples were tested in the experiment, among which the difference between the highest and lowest gray values of the background increased from 74 to 132. Minimal gray value of background
Maximal gray value of background
Gray value difference of background
109
183
74
1099
103
192
89
1062
96
200
104
1071
87
206
119
1078
80
212
132
1024
6
Mura image
Binary image
Area (pixel)
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
Table 5 Robustness assessment under various image sizes. Three sizes of samples were tested in the experiment, among which the background conditions are the same as Table 4. Image size (pixel)
250 × 250 500 × 500 750 × 750
Mura area (pixel) Sample 1
Sample 2
Sample 3
Sample 4
Sample 5
1099 4252 10028
1062 4296 9551
1071 4327 9698
1078 4319 9229
1024 4131 9746
(3)
R = C1/ C2
27.5445 71.8971 261.3217
0.04% 0.03% 0.04%
During the inspection process, the size of the image and the lighting conditions may affect the detection results. Two sets of experiments were performed to evaluate the robustness of the proposed method. In order to verify the impact of the lighting environment on the experimental results, we set up the first set of experiments. Five samples of 250 × 250 pixels were tested in the experiment, among which the difference between the highest and lowest gray values of the background increased from 74 to 132. The detection results are shown in Table 4 and we calculated the area of the mura segmented from the background. The standard deviation of these data is 27.54 pixels according to Eq. (5) and relative standard deviation (RSD) is 0.04% according to Eq. (6), which demonstrates that the difference in detection results is tiny in different lighting environments. In order to verify the impact of the image size on the experimental results, we set up a second set of experiments. The images of 500 × 500 pixels and 750 × 750 pixels were tested in the same environment as the first set of experiments. The detection results are shown in Table 5 and we can find that the relative standard deviation of the test results is stable at 0.03%–0.04%. According to the above experiment, we can draw this conclusion that the proposed method has high robustness.
4.2.2. Misclassification error To evaluate the accuracy of the proposed method, ten gray-value levels of artificial mura defects were added to an LCD panel that had even gray distribution of background, in which the differences in gray values ranged from 1 to 10, as shown in Fig. 7. The misclassification error (ME) is introduced for comparison of the accuracy [12]. ME can be formulated as
BC ∩ BS + MC ∩ MS BC + MC
relative standard deviation
4.3. Robustness assessment
C1 and C2 denote the contrasts of the two methods. R varies with the sample of mura shape. We take the minimum value as the improvement ratio between the methods. The method that is proposed in this paper effectively improved the mura defect-background contrast by at least 14 times compared to the conventional DCT method and at least 2 times compared to the polynomial fitting method.
ME = 1 −
standard deviation
(4)
where BC and MC denote the correct background and mura of the artificial image, respectively, and BS and MS denote the background and mura pixels in the segmented image, respectively. The values of ME are listed in Table 2. The lower the value of ME is, the higher the accuracy of the detection is. From Table 2, we conclude that the polynomial fitting method has a good detection effect and high accuracy. In the case of a large gray difference between the background and the mura, the DCT method performs well. However, with the decrease of the gray difference, the detection effectiveness worsens and the accuracy decreases. The proposed method outperforms the other methods.
n
σ=
∑i = 1 Δsi2 n−1
RSD = σ / s
(5) (6)
where Δsi = si − s , si denotes the area of the mura in each background and s denotes the average area of all muras. These areas are in units of pixel. The number of test images is n.
5. Conclusions
4.2.3. Computation time It is significant to compare the proposed method, the DCT method and the polynomial fitting method in terms of efficiency. The value of ME and the computation time are employed to evaluate the efficiency of the algorithms. The smaller the value of ME and the shorter the run time, the higher the efficiency. ME is calculated in Table 2 and the computation times of the three methods are listed in Table 3. Six groups of experiments were performed on a personal computer with an Intel CPU of 2.5 GHz and 8 GB RAM. The image sizes of these six groups of experiments were 250 × 250 pixels, 500 × 500 pixels, 750 × 750 pixels, 1000 × 1000 pixels, 2000 × 2000 pixels, 3000 × 3000 pixels respectively. In each group of experiments, a set of 20 images were tested to calculate the average computation time. The DCT method, the proposed method and the polynomial fitting method respectively take 28 ms, 41 ms and 65 ms to complete the detection for dealing with the test images of 250 × 250 pixels. The average computation time of the three methods are shown in Fig. 8 and the proposed method is between the DCT method and the polynomial fitting method. As the size of the image increases, the computation time of all algorithms increases. However, the computation time of the polynomial fitting method has the fastest growth rate and is almost 2.5 times as our method in processing images of 3000 × 3000 pixels.
This paper proposed a mura defect detection method that is based on DCT and the dual-γ piecewise exponential transform. Aiming at improving the mura defect detection rates and detection results, the main components of a background are extracted in the frequency domain for reconstruction. Since the background cannot be completely filtered out, Otsu's method is adopted to determine the threshold as the segmentation point, while the mura defect area is enhanced by means of the dual-γ piecewise exponential transform. Lastly, Otsu's method is used for defect detection through image segmentation. In addition, to compare the mura defect detection results of the DCT method, the polynomial fitting method and the proposed method, three indices, namely, enhancement, misclassification error, and computation time, are introduced for quantitative evaluation. Compared to the traditional DCT background reconstruction method and the polynomial fitting method, the proposed method increases the contrast of the mura defect area by at least 14 times and 2 times, respectively. In addition, the method improves the accuracy of mura defect identification and can complete the detection of a 250 × 250-pixel image efficiently in 41 ms. Moreover, the robustness assessment experiments demonstrate that the proposed method has high robustness. 7
Precision Engineering xxx (xxxx) xxx–xxx
S. Jin et al.
[6] Tsai DM, Hung CY. Automatic defect inspection of patterned thin film transistorliquid crystal display (TFT-LCD) panels using one-dimensional Fourier reconstruction and wavelet decomposition. Int J Prod Res 2005;43(21):4589–607. [7] Chen SL, Chang JH. TFT-LCD Mura defects automatic inspection system using linear regression diagnostic model. Proc Inst Mech Eng Part B J Eng Manufact 2008;222(11):1489–501. [8] Bi Xin. Machine vision inspection method of Mura defect for TFT-LCD. J Mech Eng 2010;46(12):13–9. [9] Zhang Yu, Zhang Jian. Automatic blemish inspection for TFT-LCD based on polynomial surface fitting. Opto-Electronic Eng 2006;33(10):108–14. [10] Li Kun, Li Hui, et al. Background suppression of LCD Mura defect using B-spline surface fitting. Opto-Electronic Eng 2014;41(2):33–9. [11] Chen LC, Kuo CC. Automatic TFT-LCD mura defect inspection using discrete cosine transform-based background filtering and 'just noticeable difference' quantification strategies. Meas Sci Technol 2008;19(1):015507. [12] Chen LC, Chien CH, Nguyen XL. An effective image segmentation method for noisy low-contrast unbalanced background in Mura defects using balanced discrete-cosine-transfer (BDCT). Precis Eng 2013;37(2):336–44. [13] Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 1979;9(1):62–6. [14] Xie Rui, Li Gang. High quality background modeling of LCD-Mura defect. J Comput Appl 2016(04):1151–5.
Acknowledgement This work was supported by Ministry of Science and Technology of the People's Republic of China (Grant No.2013YQ220749). References [1] Lee JY, Yoo SI. Automatic detection of region-mura defect in TFT-LCD. IEICE Trans Info Syst 2004;87(10):2371–8. [2] Tseng DC, Lee YC, Shie CE. LCD mura detection with multi-image accumulation and multi-resolution background subtraction. Int J Innov Comput Inf Contr 2012;8(7):4837–50. [3] Oh JH, Kwak DM, Lee KB, et al. Line defect detection in TFT-LCD using directional filter bank and adaptive multilevel thresholding. Key Eng Mater 2004;270–273:233–8. [4] Taniguchi K, Ueta K, Tatsumi S. A mura detection method. Pattern Recogn 2006;39(6):1044–52. [5] Song YC, Choi DH, Park KH. Morphological blob-mura defect detection method for TFT-LCD panel inspection.Knowledge-based intelligent information and engineering systems. International conference, Kes. 2004.
8