- Email: [email protected]
Accurate and robust crack detection using steerable evidence filtering in electroluminescence images of solar cells
Optics and Lasers in Engineering 118 (2019) 22–33
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Accurate and robust crack detection using steerable evidence filtering in electroluminescence images of solar cells Haiyong Chen a,b,∗, Huifang Zhao a, Da Han a, Kun Liu a a b
School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin 300130, China State Key Laboratory for Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Hebei, China
a r t i c l e
i n f o
Keywords: Crack detection Solar cells Image processing Steerable evidence filter Segmentation
a b s t r a c t Solar cells are the key part of photovoltaic power system, and the online quality inspection of solar cells ensures conversion efficiency and usable lifetime of photovoltaic modules. However, automatic crack detection in EL images of solar cells has been a challenging task, owing to the heterogeneously textured background, low contrast between crack and surrounding background including randomly distributed crystal grains, the diversity of crack types, and so on. To address these challenges, this paper presents a new accurate and robust crack detection scheme for multicrystalline solar cells. Firstly, a novel steerable evidence filter is developed to generate the crack saliency map, which significantly enhances the contrast between crack and surrounding background. Secondly, a segmentation-based method including local threshold and minimum spanning tree is employed, which ensures the extraction of complete crack. Next, the crack can be accurately located in the inspection image by computing the crack skeleton. Finally, experimental results on defective and defect-free EL images show that the proposed scheme is accurate and robust, and various types of cracks can be correctly detected. The proposed scheme achieved average detection rate of 94.4% on the data set, which performs better than the previous methods.
1. Introduction Solar energy is one of the most important renewable energy sources, and solar power has become the focus of sustainable development of energy and environment in all countries of the world [1]. As the key part of photovoltaic (PV) power system, solar cells can convert solar energy into electricity. However, crack is inevitably generated in many fabrication and installation links due to the fragile nature of crystal structure [2]. The presence of crack will cause finger interruption and obstruct current transmission, which makes partial or total failure of solar cells and finally affects the stability of PV power system [3]. Therefore, the quality of solar cells is crucial for the PV industry. However, the current manual detection method lacks rapidity, reliability, and robustness for mass production of solar cells, which involves experienced technicians reviewing the solar cells images and identifying cracks. Moreover, it is a quantitative sampling method rather than a fully automatic detection for all solar cells products. Hence, computer vision-based techniques for automatic crack detection in solar cells are emerging. In practical application, solar cells images are the main data source for crack detection. However, some tiny cracks are inside the wafer surface. It is difficult for the conventional CCD camera to capture efficacious crack information in the visible light range. Thus, we acquire the solar cells images by using the electroluminescence (EL) imaging ∗
technique [4,5], which can capture the near infrared image with a wavelength of 950 nm–1250 nm in short interval times. Compared with the defect-free regions that have higher conversion efficiency, crack defects appear as dark features with curvilinear and complicated geometric structures in EL images. Fig. 1(a) is a defect-free EL image and there are crystal grains with random shapes, sizes, positions and orientations forming the heterogeneously textured background. Specially, as shown in the red label frames of Fig. 1(b) and (c), the cracks are submerged by the randomly distributed crystal grains, showing a lower contrast between crack and surrounding background. Moreover, some crystal grains also appearing as curvilinear shapes may be mistaken for cracks. Therefore, automatic crack detection has always been a challenging task in the field of defect detection. In recent years, many methods based on computer vision techniques have focused on the crack detection in road, bridge, steel, and solar cells surfaces. Generally, current crack detection methods can be divided into three categories: spatial-, spectral-, and classification-based. Specifically, spatial domain methods mainly compare the pixel intensity differences between crack and background. Nhat-Duc [6] proposed an intensity adjustment method to enhance crack images in building structures, but it fails to detect some thin cracks due to low intensity contrast. Similarly, the intensity information for pavement crack detection was used in ref [7,8], but the background illumination and texture
Corresponding author. E-mail address: [email protected] (H. Chen).
https://doi.org/10.1016/j.optlaseng.2019.01.016 Received 12 June 2018; Received in revised form 9 January 2019; Accepted 30 January 2019 0143-8166/© 2019 Elsevier Ltd. All rights reserved.
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 1. Challenges of crack extraction in solar cells. (a) Defectfree EL image with randomly distributed crystal grains. (b) and (c) Crack defective images.
affect crack segmentation performance. Cinar et al. [9] applied phase congruency to detect crack, and Hough transform was used to segment crack. However, this method is not applicable to high curvature and complex crack detection due to the limitation of Hough transform. For crack detection in solar cells, Tsai et al. [10] applied an anisotropic diffusion scheme that took the gray-level and gradient features to adjust the diffusion coefficient. Chiou et al. [11] proposed a local thresholdbased crack extraction method and Ko et al. [12] first used histogram equalization to increase the gray level difference between crack and background, and then anisotropic diffusion also was applied to smooth image. However, for these methods, the intensity information is the major consideration, but the intensity of crystal grains is very similar to crack in solar cells. Therefore, it is difficult for the accurate extraction of crack. Spectral domain methods usually use a set of filters to process the image and obtain the features that distinguish crack from the background according to the filter response. Xu et al. [13] applied two-dimensional Haar wavelet transform to detect cracks in steel surface. Salman et al. [14,15] used the Gabor filter to detect pavement cracks. And Medina et al. [16] employed the Gabor filter invariant to rotation for crack detection in concrete tunnels. However, these methods are applied to detect transverse or longitudinal cracks in uniform background such as steel surface and do not involve crack with complicated structures. Tsai et al. [17] proposed the Fourier image reconstruction technique to detect crack defects in solar cells. This method described crack as line-shaped and removed them by setting the corresponding frequency components to zero. Thus, it is also not effective in detecting crack with complicated geometry. Ouma and Hahn [18] used the triple-transform approach based on discrete wavelet transform to detect the linear crack in asphalt pavement. However, the wavelets are anisotropic and are not suitable for extracting high curvature cracks. Some learning methods apply a classifier to perform defects classification based on the extracted crack features. Zhang et al. [19] first implemented crack segmentation, then feature extraction and crack classification by Extreme Learning Machine (ELM) classifier. Prasanna et al. [20] computed multiple visual features and proposed a spatially tuned robust multifeature (STRUM) classifier for crack detection on bridge decks. Anwar and Abdullah [21] analyzed crack shape and applied support vector machine (SVM) to implement shape classification. Demant et al. [22] proposed a crack detection algorithm for PL- and IR-images of silicon wafers. A set of labeled crack images were used to train to obtain local descriptor, and the cracks can be identified by the support vector machine classifier. Different from the manually designed features in classification methods, the deep learning algorithm has a powerful automatic feature extraction capability and has become a hot research topic. In [23], deep convolutional neural network (CNN) was used to automatically learn features from manually labeled image patches and then determine if the test image has crack defects. Chen and Jahanshahi [24] applied CNN to detect crack patches from inspection videos of nuclear power plants. All the above classification methods and the deep learning methods require a large number of labeled images and training data to ensure better detection performance. Moreover, they can only distinguish defective
images and defect-free images, so the accurate location of a complete polymorphic crack in complicated surface is unachievable. Although current research methods have achieved certain results, automatic crack detection in EL images of solar cells is still demanding for the following reasons. (1) Current methods fail to solve the heterogeneously textured background interferences; (2) current methods are inefficient in dealing with crack in low contrast background, leading to an incomplete detection; and (3) most of current methods are proposed to detect line-shaped cracks, so complex cracks with sharp bends and bifurcations are not well solved. In this paper, we propose an automatic crack detection scheme that can solve the above challenging problems. The remaining part of this paper is organized as follows. Section 2 presents the crack detection system for solar cells. In Section 3, an accurate and robust crack detection scheme for multicrystalline solar cells is explained concretely. Section 4 describes the experimental results on a series of defective and defect-free EL images. Finally, Section 5 gives the conclusion. 2. Crack detection system for solar cells 2.1. Image capture and processing Fig. 2 describes the total configuration of crack detection system in our lab. The EL images acquisition and processing system of solar cells mainly include three units: the image acquisition unit, electrode holder and motion unit, and transmission unit. Among them, the most important part is the image acquisition unit that ensures the image with defect information is captured. Specially, the EL images data of solar cells is obtained in a darkroom in order to avoid interference from ambient visible light. The image acquisition unit is composed of the near infrared Mono Chrome camera of STC-SBS500POE with resolution of 5.0 million (2448 × 2048) and Satoo industrial lens of VTG1214-M4. The electrode holder and motion unit contain probe, copper electrode, bracket and vertical movement device driven by the servo motor. In addition, the transmission unit contains a servo motor and a conveyor belt. The central controller PLC controls these three units. When the sensor senses that the solar cell is under the probe, it will send a presence signal to the PLC. Then, the PLC sends a stop motion signal to the conveyor belt. After the conveyor belt stops moving, the PLC sends a probe down signal and the probe is lowered. After the probe stops at the set position, the image acquisition signal is immediately sent to camera that acquires EL images during the set exposure interval. After image acquisition is complete, the probe rises and the conveyor belt moves, and then an image acquisition process in the T1580 workstation computer is completed. In addition, the T1580 workstation containing 3.5 GHz high-performance processor with 32GB memory ensures real-time online crack detection of solar cells. 2.2. EL images and difficulties analysis of crack detection The defect-free EL images have no crack defects but crystal grains randomly distributed in the background. The defective EL images can 23
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 2. Diagrammatic sketch of the crack detection system. (a) and (b) Parts of the hardware. (c) Procedure of crack detection.
Fig. 3. Representative defect-free images and five different types of defective images: (a1) and (a2) Pure-type cracks. (b1)–(b5) Submerged-type cracks. (c1) and (c2) Dendritic-type cracks. (d1) and (d2) Break defects. (e1) and (e2) Dark-type defects. (f1) and (f2) Defect-free.
be divided into five types according to the texture background and geometric property of crack. Specifically, pure-type cracks have relatively uniform background and no crystal grains near the cracks. Submergedtype cracks have some portions submerged by the crystal grains with similar intensity. Dendritic-type cracks have complicated structures, and break defects appear as a boundary that divides the EL image into a light region and a dark region. Dark-type cracks happen in the low-efficiency solar cells, which show low contrast between crack and background in the corresponding EL images. Some representative defective and defectfree EL images are shown in Fig. 3. As mentioned in the previous section, the difficulties of crack detection in EL images mainly are (1) the heterogeneously textured background caused by the random crystal grains; (2) low contrast between crack and surrounding background including crystal grains; and (3) the
diversity of crack types. The crack detection difficulties in EL images can be shown in Fig. 4. Fig. 4(a) is a defective EL image where crystal grains are distributed near the crack. Two defective image patches are selected from the EL image. Specially, the red-labeled image patch includes crack with crystal grains around it, and the crack seems to be submerged by the crystal grains. In contrast, the green-labeled image patch contains crack in a uniform background and there is no crystal grains interference around the crack. The corresponding gray level intensities in two patches are shown in Fig. 4(b). The result shows crystal grains and crack have similar intensity distribution in image patch labeled as red, which greatly increases the difficulty of crack detection. The selected crystal grains labeled as yellow also present curvilinear shapes, which may cause crystal grains easy to be mistaken for crack information. 24
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 4. Difficulties analysis results of crack detection in EL images. (a) Defective EL image and selected patches. (b) Corresponding gray levels intensity, in which crack with crystal grains is labeled as red and the single crack patch is labeled as green. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 5. Total scheme of the proposed method.
3. Crack detection scheme
( 2 ) ( ) 𝑥 − 𝜎2 ( ) 𝑥2 + 𝑦2 𝑔𝑥𝑥 𝑝0 = √ 𝑒𝑥𝑝 − 2 2𝜎 2𝜋𝜎 5 ( ) ( ) 𝑥𝑦 𝑥2 + 𝑦2 𝑔𝑥𝑦 𝑝0 = √ 𝑒𝑥𝑝 − 2𝜎 2 2𝜋𝜎 5 ( 2 ) ( ) 𝑦 − 𝜎2 ( ) 𝑥2 + 𝑦2 𝑔𝑦𝑦 𝑝0 = √ 𝑒𝑥𝑝 − 2𝜎 2 2𝜋𝜎 5
The proposed accurate and robust crack detection scheme includes three steps, which are crack saliency map generation by steerable evidence filtering; segmentation-based crack extraction by local threshold and minimum spanning tree; and finally crack location by the skeleton extraction. The total scheme is shown in Fig. 5. Firstly, a novel steerable evidence filter is developed to process EL image to generate the crack saliency map, which can enhance the contrast between crack and surrounding background and provide evidence for the presence of crack. Secondly, a segmentation-based crack extraction method is employed to extract complete crack. In this process, a local threshold based on sliding sub-image is utilized to segment crack from the crack saliency map. Then, the morphology operation is used to remove some isolated non-crack pixels and minimum spanning tree is applied to connect crack fragments. Finally, the accurate and complete crack can be located in the inspection image by computing crack skeleton.
(3)
(4)
(5)
Considering the orientation 𝑢𝜃 = (cos 𝜃, sin 𝜃)𝑇 , the basic steerable filter with 𝜃 ∈ [−𝜋∕2, 𝜋∕2] can be obtained by Eq. (6). And its detailed expression is shown in Eq. (7). ) ( ( ) 𝑒 𝑝0 , 𝜃; 𝜎 = 𝑢𝑇𝜃 𝐻 𝑝0 𝑢𝜃 (6) ) ( 𝑒 𝑝0 , 𝜃; 𝜎 = 𝑔𝑥𝑥 𝑐𝑜𝑠2 𝜃 + 𝑔𝑦𝑦 𝑠𝑖𝑛2 𝜃 + 𝑔𝑥𝑦 𝑠𝑖𝑛2𝜃
(7)
After the basic steerable filter e convolutes an image f(p), the filter response E at a position p0 can be calculated by Eq. (8). In addition, the scale size 𝜎 in the steerable basic filter can be fixed to adapt different width. As shown in Fig. 6(a), the first row shows three basic steerable filters with 𝜃 = 0, −𝜋∕4, 𝜋∕3, respectively. ( ( ( ) ) ) (8) 𝐸 𝑝0 , 𝜃; 𝜎 = 𝑒 𝑝0 , 𝜃; 𝜎 ∗ 𝑓 𝑝0
3.1. Crack saliency map generation by steerable evidence filtering The steerable filters derived from a linear combination of basic filter with arbitrary orientations [25]. In this study, the basic filter generates from Hessian Matrix H(p), which is a square matrix and describes the local curvature of the function. For a 2-D image f(p), at a position 𝑝0 = (𝑥, 𝑦), the hessian function can be obtained by Eq. (1). The sign ′∗ ′ represents a convolution operation. ( )) ( ( ) ( ) ( ) 𝑔𝑥𝑥 𝑝0 𝑔𝑥𝑦 𝑝0 ( ) ( ) ∗ 𝑓 𝑝0 𝐻 𝑝0 = (1) 𝑔𝑦𝑦 𝑝0 𝑔𝑥𝑦 𝑝0
However, the single basic steerable filter is incapable of detecting sharp bends, intensity variations, and complicated crack morphology. Thus, inspired by the local directional evidence filter [26], we design two additional oriented filters that include a certain offset in the angle and space distance of the detection point. Eqs. (9) and (10) are the corresponding two offset point p1 and p2 . Fig. 6(b) is the demonstration of two additional oriented filters. It shows a local search region around the detection point 𝑝0 (𝑥, 𝑦) = (1, 1), and the parameter settings are 𝑑 = 1, 𝜃 ∈ [−𝜋∕2, 𝜋∕2] and 𝜑 ∈ [−𝜋∕6, 𝜋∕6] with step of 𝜋/18. The yellow point
Eqs. (2)–(5) give the Gaussian kernel with variance 𝜎 and its corresponding second derivatives. ( ) ) ( 𝑥2 + 𝑦2 1 𝑔 𝑝0 ; 𝜎 = √ (2) 𝑒𝑥𝑝 − 2𝜎 2 2𝜋𝜎 25
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 6. (a) Example of the basic steerable filters and the SEFs with different orientations, respectively. (b) The local search region of the two additional oriented filters with offset point p1 and p2 . (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article)
represents the detection point p0 , and the red points represent the p1 , and p2 is displayed with green points, all of which appear as a circular region with the center of detection point p0 and the radius of d. Specially, the red points and green points partially overlap caused by the offset angle 𝜑 and the offset distance d, which ensure the detection of cracks at the entire circular region in the detection point. Thus, as shown in Eq. (11), the linear superposition of the basic steerable filter and two additional oriented filters forms the steerable evidence filter (SEF). As shown in Fig. 6(a), the second row shows three steerable evidence filters with different orientations. It indicates the SEF is consistent with the geometric characteristics of the crack fragments including line and curve, so the problems of crack bending, bifurcation and complicated geometry can be addressed. 𝑝1 = [𝑥 − 𝑑 cos (𝜃 + 𝜑), 𝑦 + 𝑑 sin (𝜃 + 𝜑)]
(9)
𝑝2 = [𝑥 + 𝑑 cos (𝜃 + 𝜑), 𝑦 − 𝑑 sin (𝜃 + 𝜑)]
(10)
( ( ) ( ) ) 𝑒∗ (𝑝; 𝜎, 𝜃, 𝜑) = 𝑒 𝑝0 ; 𝜎, 𝜃 + 𝑒 𝑝1 ; 𝜎, 𝜃 + 𝜑 + 𝑒 𝑝2 ; 𝜎, 𝜃 + 𝜑
(11)
paper, the threshold for segmenting crack from the crack saliency map is given by Eq. (14) 𝑇 (𝑥, 𝑦) = 𝜇𝑠 (𝑥, 𝑦) + 𝑘𝜎𝑠 (𝑥, 𝑦)
where 𝜇 s (x, y) and 𝜎 s (x, y) are the mean and standard deviation of each sliding sub-image of size N × N in S(x, y), and the size of mask is set 9 × 9 in our application. k is a predetermined constant. We select three EL images, and the corresponding segmentation results are given by Eq. (15). Although some background pixels are removed after the threshold process, there are still some crystal grains structures in the threshold segmentation result, so morphological operation is adopted to remove some small non-crack pixels and the final segmentation results are shown in Fig. 8. However, for these EL images with crystal grains interference, the segmented crack is disconnected near the crystal grains, which makes crack incomplete, as shown in Fig. 8(b) and (c). { 1, 𝑖𝑓 𝑆 (𝑥, 𝑦) > 𝑇 (𝑥, 𝑦) 𝐵 (𝑥, 𝑦) = (15) 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 To get a complete crack, minimum spanning tree (MST) based on Kruskal’s algorithm is employed. Minimum spanning tree is an important model in graph theory and it is used to solve the problem of minimal path cost [27]. As an illustration, Fig. 9(a) includes some nodes that have undirected connected graphs, and Fig. 9(b) is the complete tree by minimum spanning tree. In this process, minimum spanning tree can minimize the sum of the weights of corresponding edges and connect these nodes. After that, we use minimum spanning tree to connect crack fragments. Fig. 9(c) shows some crack fragments, which are caused by the crystal grains submerge the crack. The nodes of a1 and b1, c1 and d1 are the vertexes of crack fragments to be connected, respectively, and Fig. 9(d) shows the complete cracks after minimum spanning tree operation. In order to mark the crack defects, we compute the crack skeleton and locate them in the EL images. As shown in Fig. 9(e), the complete crack defects marked as red are accurately located.
In our application, we aim at finding the crack that lies in some uncertain position and orientation in an EL image. When convoluting an image with the SEF in Eq. (12), higher response magnitudes will be obtained in the crack region. To better highlight crack information, the maximum response magnitudes R∗ are calculate by Eq. (13), which form the crack saliency map S(x, y) for subsequent crack extraction. Fig. 7 shows three crack EL images and the corresponding saliency maps generated by the SEF. In addition, we focus on the different response of a defect-free patch and a crack patch. It shows the response magnitudes of defect-free region can be approximately treated as zero. In contrast, the crack region has higher response magnitudes, which makes crack salient and contributes to the further crack acquirement. 𝑅(𝑝; 𝜎, 𝜃, 𝜑) = 𝑒∗ (𝑝; 𝜎, 𝜃, 𝜑) ∗ 𝑓 (𝑝)
(12)
𝑅∗ (𝑝; 𝜎) = max 𝑅(𝑝; 𝜎, 𝜃, 𝜑)
(13)
𝜃,𝜑
(14)
4. Experimental results and discussion
3.2. Segmentation-based crack extraction from the saliency map
4.1. Parameter setting
In this section, we extract crack from the saliency map S(x, y) based on segmentation method. Owing to the random crystal grains, the defect-free region also presents certain response magnitudes in the saliency map. In order to extract the crack, we first apply a local threshold based on sliding sub-image to segment crack from the saliency map. Then, according to Tsai’s methods [10,17], they used a simple statistical control limit method to determine the threshold. Similarly, in this
In order to verify the performance of the proposed method, we collect the defective and defect-free EL images captured by the infrared camera on the production line in Tianjin Yingli new energy Co., Ltd. Here, 372 defective including different types’ cracks and 200 defect-free EL images are used to verify the performance of the SEF method. Although there are several parameters in the proposed method, not all of them play important roles in crack detection performance. In 26
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 7. Example of the crack saliency map. (a) Original crack EL images. (b) The saliency maps generated by the SEF. Fig. 8. Crack fragments obtained by local threshold in the saliency map of EL images. (a) Complete crack. (b) and (c) Crack fragments.
Fig. 9. Demonstration of minimum spanning tree and crack fragments connection. (a) Nodes to be connected. (b) Complete tree. (c) Crack fragments. (d) Connected crack. (e) Located crack.
27
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 10. Crack detection performance changes with different parameters values. (a)–(d) Four defective EL images. (a1)–(a3) Effect of scale size with 0.5, 1 and 2. (b1)–(b3) Effect of offset with 0.5, 1 and 2. (c1)–(c3) Effect of threshold k with 0.5, 1 and 1.5. (d) Effect of sub-image of size × N with 5 × 5, 9 × 9 and 17 × 17.
Table 1 Detail of the data set used in the experiment.
Types
Number of test images
Number of defective images used for optimal parameters selection
Pure crack Submerged crack Dendritic crack Break Dark crack Defect-free
113 170 17 49 23 200
20 20 10 10 10 20
our proposed scheme, four parameters are essential: SEF scale size 𝜎, offset d, threshold coefficient k and sub-image of size N × N. Fig. 10 shows the crack detection performance changes with different parameter values. The optimal parameter values in EL images of solar cells can be selected by assessing these results [17]. In this paper, in order to ensure the effectiveness of the selected optimal parameter values, we also apply some extensive experiments on selected a certain number of defective and defect-free EL images in Table 1. These extensive experiments have similar detection performance to Fig. 10. Moreover, the optimal detection results on selected EL images are counted under different parameter values, which are presented in Fig. 11. Concretely, the ordinate in Fig. 11 presents the number of EL images that can achieve the optimal detection performance of Fig. 10. For example, as shown in Fig. 11(a), the pink rectangle in pure crack type indicates that the crack can be accurately and completely extracted in all the 20 selected defective EL images when the scale size is set to 1. Particularly, all the 20 detection results can be achieved the optimal detection performance of Fig. 10(a2). Similarly, the blue rectangle in break type of Fig. 11(b) shows that altogether 7 defective EL images obtain the detection performance of Fig. 10(b2). In addition, all the detection results are achieved when the offset is set to 0.5. Other detection results of different parameters values in Fig. 11 are also obtained in this way. Fig. 11 presents the detection results according to different parameters values for the selected EL images. In the steerable evidence filtering procedure, we have to determine two parameters: the scale size 𝜎 and the offset parameter d. The parameter 𝜎 fits different widths of crack defects. As shown in Fig. 10(a1)–(a3), if 𝜎 is set too small, the crack cannot be well highlight and it is easy to be missed. Conversely, if 𝜎 is set too large, the crystal grains near the crack will be treated as crack and affect the detection accuracy. As shown in Fig. 11(a), for different types of defective and defect-free images, a better detection results can be obtained when the scale size 𝜎 is 1. Similarly, as shown in Fig. 10(b1)– (b3), the offset parameter d controls the locality of the steerable evidence filter. While a small value of d does not contribute enough, a large value will introduce false pixels that do not belong to the same crack
Size of each test image (pixels by pixels) 125 125 125 125 125 125
× × × × × ×
125 125 125 125 125 125
structure. According to Fig. 11(b), the optimal performance is achieved when d is 1. Besides, the threshold coefficient k will affect the detection results in the segmentation-based crack extraction process. As shown in Fig. 10(c1)–(c3), the small k gives a tight threshold and may identify the background pixels as crack pixels. However, large k gives a loose threshold and may miss some true defect pixels. As shown in Fig. 11(c), although the same detection results can be obtained for dendritic-type crack images and defect-free images when k is 1 or 1.5, the k is set to 1 for the whole types of EL images. Additionally, the size N × N of the subimage in the local threshold process plays a certain role in the final crack segmentation results. As shown in Fig. 10(d1), if the value of N is too small, the mean and standard deviation are calculated in a small image region. It includes more local details such as crystal grains, making it difficult to distinguish crack from these crystal grains. Therefore, crack is easily removed in the subsequent processing. Conversely, as shown in Fig. 10(d3), if the value of N is too large, some non-crack background pixels are easily to be identified as crack fragments. Figs. 10(d2) and 11(d) show the optimal performance is achievement when the size of sub-image is 9 × 9.
4.2. Robustness analysis in brightness levels In this study, we also test some dim defective images that present low contrast between crack and background and defect-free images, and the proposed method still achieve satisfactory performance. Fig. 12(d1) is a dim defect-free EL image, and Fig. 12(a1)–(c1) are three typical low-efficiency solar cells that show large dark areas, causing the weak differences between crack defect and background in the corresponding EL images. The corresponding crack saliency maps are shown in Fig. 12(a2)–(d2), in which crack defects become salient. The detection result of Fig. 12(d3) shows that no crack is detected in the defect-free EL image. The crack defects still can be complete and accurately located by red curves in the dim defective EL images simultaneously. Therefore, the proposed method is robust to different brightness levels. 28
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 11. Optimal parameter values selection. (a) Effect of scale size. (b) Effect of offset d. (c) Effect of threshold k. (d) Effect of sub-image of size N × N. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article)
4.3. Performance evaluation
gradient in EL images. Moreover, the crack detection results are intermittent, such as in Fig. 13(a2) and (e2). Specially, this method is less adept at detecting the low contrast crack in the dim EL images, which is shown in Fig. 13(f2). The Chiou’s method uses a local threshold-based algorithm to segment crack. It mainly partitions an image into several sub-images and calculates each sub-image’s mean and standard deviation. The detection results are shown in Fig. 13(a3)–(g3), and the crystal grains are detected as crack due to only take intensity information into account. In addition, the crack defect cannot be detected in the dim EL images, such as in Fig. 13(f3). The Anwar’s method applies an improved anisotropic diffusion filtering with intensity-based diffusion coefficient, which can enhance the crack pixels with low gray level and high gradient. In addition, shape analysis is performed to distinguish between crack and other arbitrary patterns. To get optimal detection performance, parameter b is set to 1, and the iteration t is set to 4 in this paper. The corresponding detection results are shown in Fig. 13(a4)–(g4). Concretely, Fig. 13(b4) shows some random crystal grains also appear as low intensity and high gradient features. These crystal grains show crack-shaped in the segmentation results, which are easily to be identified as crack. In addition, for low contrast defective image such as Fig. 13(f4), this crack detection method achieves poor performance due to the weak intensity and gradient differences between crack and surrounding background. In contrast, for defect-free image in Fig. 13(g4), this method has good robustness.
4.3.1. Qualitative evaluation In this paper, our SEF method is compared with the Tsai’s method [10], Chiou’s method [11], Anwar’s method [21], and the basic steerable filter (BSF) method [25] on the given image data set. The performance of four comparison methods may be not fully effective relative to the original work due to the changes in the data set. However, we have adjusted some key parameters to make these methods more suitable for the current data set and ensure a relatively high detection rate. As an illustration, Fig. 13 shows representative six defective EL images, one defect-free EL image and the corresponding detection results. The first row shows the original images and the corresponding detection results are given from the second column to sixth column, respectively. The seventh column is the manually labeled ground truth images of crack defect. The Tsai’s method assumes that the crack defect presents the features of low gray level and high gradient. In that case, the anisotropic diffusion model will generate high diffusion coefficients to smooth crack defect and preserve the gray level of defect-free background, simultaneously. In order to get better detection performance, we set the diffusion iterations 𝑇 = 3, regularization value 𝑘 = 4, and control constant 𝑐 = 2, and the detection results are shown in Fig. 13(a2)–(g2). Experimental results show that it mistakes some crystal grains as crack defect because the random crystal grains also exhibit features of low gray level and high 29
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 12. Demonstrative luminance robustness of the SEF method.
The BSF method uses the single basic steerable filter to process the EL images. Fig. 13(a5) shows that the BSF is less adept at solving intensity variations, making the crack incomplete. Moreover, the BSF method cannot well address the bifurcations due to the lack of consideration of local neighborhood information. As shown in Fig. 13(d5), the crack in the bifurcation is missed. However, for the dim EL images, this method can achieve a better result, such as in Fig. 13(f5). The detection results based on the SEF method are shown in Fig. 13(a6)–(g6). Although many crystal grains around the crack, the proposed method can detect complete crack and locate it in the inspection image. Moreover, all the false detection of crystal grains by the Tsai’s method, Chiou’s method and Anwar’s method are well addressed. Furthermore, the SEF method can well detect crack in the dim EL images and it is robust to defect-free image, such as Fig. 13(f6)–(g6). Overall, it achieves much better performance than other three methods.
Eq. (18) is the F-measure index, which can comprehensively evaluate the performance of the crack detection algorithm. The higher the value, the more effective the detection algorithm is. 𝐹 − measure =
𝐿 𝐿𝑔
(16)
𝑐𝑟𝑡 =
𝐿 𝐿𝑡
(17)
(18)
For five different types of defective EL images, the quantitative evaluation based on the statistic results is shown in Table 2. The results obtained by the Tsai’s, Chiou’s, Anwar’s, BSF method, and the SEF method are shown, respectively. For example, the first row of Table 2 shows the detection results obtained on the pure-type crack images using four different methods. The cpt, crt, and F-measure are given in different columns for each method individually. The last row shows the overall performance of different methods. From Table 2, it can be observed that the Tsai’s method has a lower cpt than the other methods owing to the similarities of gray levels and gradients between crack and crystal grains. Especially, the Tsai’s method performs badly for dark-type crack in dim EL images. Moreover, for most of the break defects, there are scarcely crystal grains interferences in the background, so the Tsai’s method achieves a better performance. In contrast, for the other crack defect types, there are randomly distributed crystal grains in the background, which leads to lower detection rate. The Chiou’s method is still unable to detect dark-type cracks because of weak difference between crack and surrounding background. Although most of crack pixels can be identified, many crystal grains are mistaken as crack and the detection results are inaccurate. The Anwar’s method cannot effectively detect dark-type cracks in low contrast images. The BSF method mainly shows poor performance on intensity variations and bifurcations. In contrast, the SEF method can get the better detection rate than the other three methods. As the more intuitive description, a graphical representation of Fmeasure based on the above five methods is shown in Fig. 14. From the experimental results in Fig. 14, it can be seen that the proposed SEF method gives the better detection results than other four methods. In particular, for break and dark-type cracks, it has a high robustness and
4.3.2. Accuracy evaluation For quantitatively estimating the performance of the proposed SEF method for multicrystalline solar cells, we apply three evaluation indices for the accuracy evaluation: completeness (cpt), correctness (crt) and Fmeasure, which are used to evaluate the crack detection performance in EL images of solar cells [21] and pavement surface [28]. Specifically, the cpt index and crt index explain the completeness and effectiveness of crack detection algorithm, respectively. The cpt and crt are calculated as shown in Eqs. (16) and (17). 𝑐𝑝𝑡 =
2 × 𝑐𝑝𝑡 × 𝑐𝑟𝑡 𝑐𝑝𝑡 + 𝑐𝑟𝑡
where Lg is the number of crack pixels in the corresponding ground truth image obtained from manual marking. L is the number of pixels in the crack detection result which matches the ground truth crack pixels. Lt is the total number of extracted pixels in the detection result. 30
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 13. Comparison of the proposed method with previous methods. (a1)–(g1) Six defective EL images and one defect-free image. (a2)–(g2) Detection results of the Tsai’s method. (a3)–(g3) Detection results of the Chiou’s method. (a4)–(g4) Detection results of the Anwar’s method. (a5)–(g5) Detection results of BSF method. (a5)–(g5) Detection results of proposed method. (a7)–(g7) Crack ground truth. All the detective regions are labeled as red.
achieves the best detection performance. Overall, the performance of the proposed SEF method is better than that of the other methods. In addition, we also qualitatively evaluate the correctness of different methods for 200 defect-free images. The detection results are summarized in Table 3. It shows the proposed method is more robust to defect-free images. In terms of computational efficiency, as shown in Table 4, for per test image with the size 125 × 125 pixels, the SEF method takes more
time than the Tsai’s method, Chiou’s method, Anwar’s method and BSF method. However, considering the crack detection results, current detection time is also tolerable. 4.4. Further analysis and discussion In steerable evidence filtering process, we can effectively obtain the crack saliency map, which includes the candidate crack pixels. 31
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Table 2 Experimental performance on 5 different types of crack defects using different methods. Types of cracks
Tsai’s method
Pure crack Submerged crack Dendritic crack Break Dark-type crack Overall performance
Chiou’s method
Anwar’s method
BSF method
SEF method
cpt
crt
Fm
cpt
crt
Fm
cpt
crt
Fm
cpt
crt
Fm
cpt
crt
Fm
37.6 64.5 53.3 80.9 38.5 55.0
63.6 84.5 80.0 82.9 33.3 68.9
47.3 73.2 64.0 81.9 35.7 60.4
54.1 79.6 58.8 94.7 31.8 63.8
97.0 94.2 100 76.6 87.5 91.1
69.5 86.3 74.1 84.7 46.4 72.2
89.1 79.3 78.2 89.2 56.7 78.5
90.2 85.4 80.9 88.5 59.4 80.9
89.6 82.2 79.5 88.8 58.0 79.6
89.3 85.9 78.8 93.6 94.5 88.4
90.4 93.0 82.5 100 95.4 92.3
89.8 89.3 80.6 96.7 94.9 90.3
95.0 88.3 92.3 97.2 96.7 93.9
91.1 94.6 89.5 100 100 95.0
93.0 91.3 90.9 98.6 98.3 94.4
Fig. 14. Variation of F-measure by different methods.
Table 3 The qualitative evaluation for defect-free images. defect-free images
Tsai’s method
200
defect-free 157
defective 43
Chiou’s method
Anwar’s method
BSF method
defect-free 139
defect-free 193
defect-free 191
defective 61
Table 4 Comparison of computational time. Methods
Times (ms)
Tsai’s method Chiou’s method Anwar’s method BSF method SEF method
72 64 83 79 86
defective 7
SEF method defective 9
defect-free 198
defective 2
(2) Crystal grains with similar intensity to cracks are mistaken as defects in Tsai’s, chiou’s and Anwar’s methods. The SEF method avoids this problem. (3) In other four methods, the detected cracks are incomplete due to inconsistent intensity and bifurcation. However, this problem can be solved by steerable evidence filter and minimum spanning tree to acquire complete crack. In addition, the SEF method has some deficiencies that need to be further exploration. First, some parameters such as the scale size 𝜎 and the offset parameter d, are experimentally determined by a certain number of test images, so it may not be the optimal value for all large of amounts of images. So, if we can exploit the correlation of these parameters to achieve the goal of adaptive adjustment, then the detection performance will be further improved. Second, after the local threshold procedure, some small fragments are similar to crack fragments. Segmented crack may miss a portion when we use the morphological operation to remove some non-crack pixels. Thus, the detection performance will be affected when some curvilinear crystal grains are in the background.
During the crack extraction process, we analyze the crack saliency map by local threshold and minimum spanning tree to acquire the complete crack. Comparing with Tsai’s and Chiou’s methods, which just use the intensity information to identify crack from the background and mistake many crystal grains as crack pixels. Anwar’s method uses improved anisotropic diffusion and segmentation technique to obtain crack defect. This method cannot remove crack-shaped crystal grains that appear low intensity and high gradient features. Especially, it is difficulty for these three methods to detect crack in the dim EL images, which show small intensity differences between crack and surrounding background. In addition, the BSF method fails to address intensity variations and bifurcations. Thus, the SEF method has three advantages over these four methods.
5. Conclusion In this paper, an automatic crack detection system for multicrystalline solar cells is presented which utilize the electroluminescence (EL) imaging technique to capture crack defect information.
(1) The crack defects in the dim EL images, which Tsai’s, chiou’s and Anwar’s methods cannot detect are well addressed by the SEF method, as this dark-type crack is not salient in the dim background. In addition, for defect-free images, the SEF method is more robust than other methods.
(1) A novel steerable evidence filter is developed to enhance the contrast between crack and background and provide evidence for the presence of crack defect. Then, local threshold and minimum 32
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
spanning tree are employed to extract complete crack. Finally, crack defect can be accurately located in the inspection image. (2) Compared with previous methods, the proposed SEF method is more robust to the heterogeneously textured background interferences, crack types and image brightness levels change. Specially, this method can well solve low contrast and complex cracks with sharp bends and bifurcations. Moreover, considering the accuracy, robustness and processing time of crack detection, the proposed SEF method can achieve satisfactory detection results and meet the practical online detection requirement.
[19] Zhang W, Zhang Z, Qi D, et al. Automatic crack detection and classification method for subway tunnel safety monitoring. Sensors 2014;14(10):19307–28. [20] Prasanna P, Dana KJ, Gucunski N, et al. Automated crack detection on concrete bridges. IEEE Trans Autom Sci Eng 2016;13(2):591–9. [21] Anwar SA, Abdullah MZ. Micro-crack detection of multicrystalline solar cells featuring an improved anisotropic diffusion filter and image segmentation technique. EURASIP J Image Video Process 2014;2014(1):15. [22] Demant M, et al. Microcracks in silicon wafers I: inline detection and implications of crack morphology on wafer strength. IEEE J Photovoltaics 2016;6(1):126–35. [23] Zhang L, Yang F, Zhang YD, et al. Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing, IEEE; 3708–3712. [24] Chen FC, Jahanshahi MR. NB-CNN: deep learning-based crack detection using convolutional neural network and naïve bayes data fusion. IEEE Trans Indust Electron 2018;65(5):4392–400. [25] Freeman WT, Adelson EH. The design and use of steerable filters. IEEE Trans Pattern Anal Mach Intell 1991;13(9):891–906. [26] Mukherjee S, Acton ST. Oriented filters for vessel contrast enhancement with local directional evidence. In: 2015 IEEE 12th International Symposium on Biomedical Imaging. IEEE; 2015. p. 503–6. [27] Graham RL, Hell P. On the history of the minimum spanning tree problem. Ann History Comput 1985;7(1):43–57. [28] Li Q, Zou Q, Zhang D, et al. FoSA: F∗ seed-growing approach for crack-line detection from pavement images. Image Vision Comput 2011;29(12):861–72.
Acknowledgement This work was supported in part by National Natural Science Foundation (NNSF) of China under grant 61873315, 61403119, Natural Science Foundation of Hebei Province under grant F2018202078, Tianjin Science and Technology Commissioner Project under grant 18JCTPJC55200, Science and Technology Program of Hebei Province under grant 17211804D, Hebei Province Outstanding Youth Science Fund (F2017202062), and Young Talents Project in Hebei Province under Grant 210003.
Haiyong Chen received the M.S. degree from the Harbin University of Science and Technology, Harbin, China, in 2005, and the Ph.D. degree from the Institute of Automation, CAS (Chinese Academy of Sciences), Beijing, China, in 2008. He is currently a Professor with the School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. His research interests include image processing, robot vision, and pattern recognition.
Supplementary materials Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.optlaseng.2019.01.016. References [1] Kabir E, Kumar P, Kumar S, et al. Solar energy: potential and future prospects. Renewable Sustainable Energy Rev 2018;82:894–900. [2] Abdelhamid M, Singh R, Omar M. Review of microcrack detection techniques for silicon solar cells. IEEE J Photovoltaics 2014;4(1):514–24. [3] Dhimish M, Holmes V, Mehrdadi B, et al. The impact of cracks on photovoltaic power performance. J Sci 2017;2(2):199–209. [4] Israil M, Anwar SA, Abdullah MZ. Automatic detection of micro-crack in solar wafers and cells: a review. Trans Inst Meas Control 2013;35(5):606–18. [5] Frazao M, Silva JA, Lobato K, et al. Electroluminescence of silicon solar cells using a consumer grade digital camera. Measurement 2017;99:7–12. [6] Nhat-Duc H. Detection of surface crack in building structures using image processing technique with an improved otsu method for image thresholding. Adv Civil Eng 2018;2018:1–10. [7] Kamaliardakani M, Sun L, Ardakani MK. Sealed-crack detection algorithm using heuristic thresholding approach. J Comput Civil Eng 2014;30(1):04014110. [8] Sun L, Kamaliardakani M, Zhang Y. Weighted neighborhood pixels segmentation method for automated detection of cracks on pavement surface images. J Comput Civil Eng 2015;30(1):04015021. [9] Cinar AF, Barhli SM, Hollis D, et al. An autonomous surface discontinuity detection and quantification method by digital image correlation and phase congruency. Opt Lasers Eng 2017;96:94–106. [10] Tsai DM, Chang CC, Chao SM. Micro-crack inspection in heterogeneously textured solar wafers using anisotropic diffusion. Image Vision Comput 2010;28(3):491–501. [11] Chiou YC, Liu JZ, Liang YT. Micro crack detection of multi-crystalline silicon solar wafer using machine vision techniques. Sens Rev 2011;31(2):154–65. [12] Ko SS, Liu CS, Lin YC. Optical inspection system with tunable exposure unit for micro-crack detection in solar wafers. Opt Int J Light Electron Opt 2013;124(19):4030–5. [13] Xu K, Xu Y, Zhou P, et al. Application of RNAMlet to surface defect identification of steels. Opt Lasers Eng 2018;105:110–17. [14] Khan H A, Salman M, Hussain S, Khurshid K. Automation of optimized Gabor filter parameter selection for road cracks detection. Int J Adv comput Sci Appl 2016;7(3):269–75. [15] Zalama E, Gómez-García-Bermejo J, Medina R, et al. Road crack detection using visual features extracted by Gabor filters. Computer-aided Civil Infrastruct Eng 2014;29(5):342–58. [16] Medina R, Llamas J, Gómez-García-Bermejo J, et al. Crack detection in concrete tunnels using a Gabor filter invariant to rotation. Sensors 2017;17(7):1670. [17] Tsai DM, Wu SC, Li WC. Defect detection of solar cells in electroluminescence images using Fourier image reconstruction. Sol Energy Mater Sol Cells 2012;99:250–62. [18] Ouma YO, Hahn M. Wavelet-morphology based detection of incipient linear cracks in asphalt pavements from RGB camera imagery and classification using circular Radon transform. Adv Eng Inf 2016;30(3):481–99.
Huifang Zhao received the B.S. degree from the Hebei University of Science and Technology, Shijiazhuang, China, in 2016. She is currently pursuing the M.S. degree in automation, School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. Her current research interests include computer vision and pattern recognition.
Da Han received the B.S. degree from the Hebei University of Science and Technology, Shijiazhuang, China, in 2016. He is currently pursuing the M.S. degree in automation, School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. His current research interests include image processing and machine learning.
Kun Liu received the M.S. degree from the Harbin Institute of Technology, Harbin, China, in 2003, and the Ph.D. degree in automation from the Tsinghua University, Beijing, China, in 2009. She is currently an Associate Professor with the School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. Her research interests include image processing, computer vision, and pattern recognition.
33
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Accurate and robust crack detection using steerable evidence filtering in electroluminescence images of solar cells Haiyong Chen a,b,∗, Huifang Zhao a, Da Han a, Kun Liu a a b
School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin 300130, China State Key Laboratory for Reliability and Intelligence of Electrical Equipment, Hebei University of Technology, Hebei, China
a r t i c l e
i n f o
Keywords: Crack detection Solar cells Image processing Steerable evidence filter Segmentation
a b s t r a c t Solar cells are the key part of photovoltaic power system, and the online quality inspection of solar cells ensures conversion efficiency and usable lifetime of photovoltaic modules. However, automatic crack detection in EL images of solar cells has been a challenging task, owing to the heterogeneously textured background, low contrast between crack and surrounding background including randomly distributed crystal grains, the diversity of crack types, and so on. To address these challenges, this paper presents a new accurate and robust crack detection scheme for multicrystalline solar cells. Firstly, a novel steerable evidence filter is developed to generate the crack saliency map, which significantly enhances the contrast between crack and surrounding background. Secondly, a segmentation-based method including local threshold and minimum spanning tree is employed, which ensures the extraction of complete crack. Next, the crack can be accurately located in the inspection image by computing the crack skeleton. Finally, experimental results on defective and defect-free EL images show that the proposed scheme is accurate and robust, and various types of cracks can be correctly detected. The proposed scheme achieved average detection rate of 94.4% on the data set, which performs better than the previous methods.
1. Introduction Solar energy is one of the most important renewable energy sources, and solar power has become the focus of sustainable development of energy and environment in all countries of the world [1]. As the key part of photovoltaic (PV) power system, solar cells can convert solar energy into electricity. However, crack is inevitably generated in many fabrication and installation links due to the fragile nature of crystal structure [2]. The presence of crack will cause finger interruption and obstruct current transmission, which makes partial or total failure of solar cells and finally affects the stability of PV power system [3]. Therefore, the quality of solar cells is crucial for the PV industry. However, the current manual detection method lacks rapidity, reliability, and robustness for mass production of solar cells, which involves experienced technicians reviewing the solar cells images and identifying cracks. Moreover, it is a quantitative sampling method rather than a fully automatic detection for all solar cells products. Hence, computer vision-based techniques for automatic crack detection in solar cells are emerging. In practical application, solar cells images are the main data source for crack detection. However, some tiny cracks are inside the wafer surface. It is difficult for the conventional CCD camera to capture efficacious crack information in the visible light range. Thus, we acquire the solar cells images by using the electroluminescence (EL) imaging ∗
technique [4,5], which can capture the near infrared image with a wavelength of 950 nm–1250 nm in short interval times. Compared with the defect-free regions that have higher conversion efficiency, crack defects appear as dark features with curvilinear and complicated geometric structures in EL images. Fig. 1(a) is a defect-free EL image and there are crystal grains with random shapes, sizes, positions and orientations forming the heterogeneously textured background. Specially, as shown in the red label frames of Fig. 1(b) and (c), the cracks are submerged by the randomly distributed crystal grains, showing a lower contrast between crack and surrounding background. Moreover, some crystal grains also appearing as curvilinear shapes may be mistaken for cracks. Therefore, automatic crack detection has always been a challenging task in the field of defect detection. In recent years, many methods based on computer vision techniques have focused on the crack detection in road, bridge, steel, and solar cells surfaces. Generally, current crack detection methods can be divided into three categories: spatial-, spectral-, and classification-based. Specifically, spatial domain methods mainly compare the pixel intensity differences between crack and background. Nhat-Duc [6] proposed an intensity adjustment method to enhance crack images in building structures, but it fails to detect some thin cracks due to low intensity contrast. Similarly, the intensity information for pavement crack detection was used in ref [7,8], but the background illumination and texture
Corresponding author. E-mail address: [email protected] (H. Chen).
https://doi.org/10.1016/j.optlaseng.2019.01.016 Received 12 June 2018; Received in revised form 9 January 2019; Accepted 30 January 2019 0143-8166/© 2019 Elsevier Ltd. All rights reserved.
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 1. Challenges of crack extraction in solar cells. (a) Defectfree EL image with randomly distributed crystal grains. (b) and (c) Crack defective images.
affect crack segmentation performance. Cinar et al. [9] applied phase congruency to detect crack, and Hough transform was used to segment crack. However, this method is not applicable to high curvature and complex crack detection due to the limitation of Hough transform. For crack detection in solar cells, Tsai et al. [10] applied an anisotropic diffusion scheme that took the gray-level and gradient features to adjust the diffusion coefficient. Chiou et al. [11] proposed a local thresholdbased crack extraction method and Ko et al. [12] first used histogram equalization to increase the gray level difference between crack and background, and then anisotropic diffusion also was applied to smooth image. However, for these methods, the intensity information is the major consideration, but the intensity of crystal grains is very similar to crack in solar cells. Therefore, it is difficult for the accurate extraction of crack. Spectral domain methods usually use a set of filters to process the image and obtain the features that distinguish crack from the background according to the filter response. Xu et al. [13] applied two-dimensional Haar wavelet transform to detect cracks in steel surface. Salman et al. [14,15] used the Gabor filter to detect pavement cracks. And Medina et al. [16] employed the Gabor filter invariant to rotation for crack detection in concrete tunnels. However, these methods are applied to detect transverse or longitudinal cracks in uniform background such as steel surface and do not involve crack with complicated structures. Tsai et al. [17] proposed the Fourier image reconstruction technique to detect crack defects in solar cells. This method described crack as line-shaped and removed them by setting the corresponding frequency components to zero. Thus, it is also not effective in detecting crack with complicated geometry. Ouma and Hahn [18] used the triple-transform approach based on discrete wavelet transform to detect the linear crack in asphalt pavement. However, the wavelets are anisotropic and are not suitable for extracting high curvature cracks. Some learning methods apply a classifier to perform defects classification based on the extracted crack features. Zhang et al. [19] first implemented crack segmentation, then feature extraction and crack classification by Extreme Learning Machine (ELM) classifier. Prasanna et al. [20] computed multiple visual features and proposed a spatially tuned robust multifeature (STRUM) classifier for crack detection on bridge decks. Anwar and Abdullah [21] analyzed crack shape and applied support vector machine (SVM) to implement shape classification. Demant et al. [22] proposed a crack detection algorithm for PL- and IR-images of silicon wafers. A set of labeled crack images were used to train to obtain local descriptor, and the cracks can be identified by the support vector machine classifier. Different from the manually designed features in classification methods, the deep learning algorithm has a powerful automatic feature extraction capability and has become a hot research topic. In [23], deep convolutional neural network (CNN) was used to automatically learn features from manually labeled image patches and then determine if the test image has crack defects. Chen and Jahanshahi [24] applied CNN to detect crack patches from inspection videos of nuclear power plants. All the above classification methods and the deep learning methods require a large number of labeled images and training data to ensure better detection performance. Moreover, they can only distinguish defective
images and defect-free images, so the accurate location of a complete polymorphic crack in complicated surface is unachievable. Although current research methods have achieved certain results, automatic crack detection in EL images of solar cells is still demanding for the following reasons. (1) Current methods fail to solve the heterogeneously textured background interferences; (2) current methods are inefficient in dealing with crack in low contrast background, leading to an incomplete detection; and (3) most of current methods are proposed to detect line-shaped cracks, so complex cracks with sharp bends and bifurcations are not well solved. In this paper, we propose an automatic crack detection scheme that can solve the above challenging problems. The remaining part of this paper is organized as follows. Section 2 presents the crack detection system for solar cells. In Section 3, an accurate and robust crack detection scheme for multicrystalline solar cells is explained concretely. Section 4 describes the experimental results on a series of defective and defect-free EL images. Finally, Section 5 gives the conclusion. 2. Crack detection system for solar cells 2.1. Image capture and processing Fig. 2 describes the total configuration of crack detection system in our lab. The EL images acquisition and processing system of solar cells mainly include three units: the image acquisition unit, electrode holder and motion unit, and transmission unit. Among them, the most important part is the image acquisition unit that ensures the image with defect information is captured. Specially, the EL images data of solar cells is obtained in a darkroom in order to avoid interference from ambient visible light. The image acquisition unit is composed of the near infrared Mono Chrome camera of STC-SBS500POE with resolution of 5.0 million (2448 × 2048) and Satoo industrial lens of VTG1214-M4. The electrode holder and motion unit contain probe, copper electrode, bracket and vertical movement device driven by the servo motor. In addition, the transmission unit contains a servo motor and a conveyor belt. The central controller PLC controls these three units. When the sensor senses that the solar cell is under the probe, it will send a presence signal to the PLC. Then, the PLC sends a stop motion signal to the conveyor belt. After the conveyor belt stops moving, the PLC sends a probe down signal and the probe is lowered. After the probe stops at the set position, the image acquisition signal is immediately sent to camera that acquires EL images during the set exposure interval. After image acquisition is complete, the probe rises and the conveyor belt moves, and then an image acquisition process in the T1580 workstation computer is completed. In addition, the T1580 workstation containing 3.5 GHz high-performance processor with 32GB memory ensures real-time online crack detection of solar cells. 2.2. EL images and difficulties analysis of crack detection The defect-free EL images have no crack defects but crystal grains randomly distributed in the background. The defective EL images can 23
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 2. Diagrammatic sketch of the crack detection system. (a) and (b) Parts of the hardware. (c) Procedure of crack detection.
Fig. 3. Representative defect-free images and five different types of defective images: (a1) and (a2) Pure-type cracks. (b1)–(b5) Submerged-type cracks. (c1) and (c2) Dendritic-type cracks. (d1) and (d2) Break defects. (e1) and (e2) Dark-type defects. (f1) and (f2) Defect-free.
be divided into five types according to the texture background and geometric property of crack. Specifically, pure-type cracks have relatively uniform background and no crystal grains near the cracks. Submergedtype cracks have some portions submerged by the crystal grains with similar intensity. Dendritic-type cracks have complicated structures, and break defects appear as a boundary that divides the EL image into a light region and a dark region. Dark-type cracks happen in the low-efficiency solar cells, which show low contrast between crack and background in the corresponding EL images. Some representative defective and defectfree EL images are shown in Fig. 3. As mentioned in the previous section, the difficulties of crack detection in EL images mainly are (1) the heterogeneously textured background caused by the random crystal grains; (2) low contrast between crack and surrounding background including crystal grains; and (3) the
diversity of crack types. The crack detection difficulties in EL images can be shown in Fig. 4. Fig. 4(a) is a defective EL image where crystal grains are distributed near the crack. Two defective image patches are selected from the EL image. Specially, the red-labeled image patch includes crack with crystal grains around it, and the crack seems to be submerged by the crystal grains. In contrast, the green-labeled image patch contains crack in a uniform background and there is no crystal grains interference around the crack. The corresponding gray level intensities in two patches are shown in Fig. 4(b). The result shows crystal grains and crack have similar intensity distribution in image patch labeled as red, which greatly increases the difficulty of crack detection. The selected crystal grains labeled as yellow also present curvilinear shapes, which may cause crystal grains easy to be mistaken for crack information. 24
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 4. Difficulties analysis results of crack detection in EL images. (a) Defective EL image and selected patches. (b) Corresponding gray levels intensity, in which crack with crystal grains is labeled as red and the single crack patch is labeled as green. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 5. Total scheme of the proposed method.
3. Crack detection scheme
( 2 ) ( ) 𝑥 − 𝜎2 ( ) 𝑥2 + 𝑦2 𝑔𝑥𝑥 𝑝0 = √ 𝑒𝑥𝑝 − 2 2𝜎 2𝜋𝜎 5 ( ) ( ) 𝑥𝑦 𝑥2 + 𝑦2 𝑔𝑥𝑦 𝑝0 = √ 𝑒𝑥𝑝 − 2𝜎 2 2𝜋𝜎 5 ( 2 ) ( ) 𝑦 − 𝜎2 ( ) 𝑥2 + 𝑦2 𝑔𝑦𝑦 𝑝0 = √ 𝑒𝑥𝑝 − 2𝜎 2 2𝜋𝜎 5
The proposed accurate and robust crack detection scheme includes three steps, which are crack saliency map generation by steerable evidence filtering; segmentation-based crack extraction by local threshold and minimum spanning tree; and finally crack location by the skeleton extraction. The total scheme is shown in Fig. 5. Firstly, a novel steerable evidence filter is developed to process EL image to generate the crack saliency map, which can enhance the contrast between crack and surrounding background and provide evidence for the presence of crack. Secondly, a segmentation-based crack extraction method is employed to extract complete crack. In this process, a local threshold based on sliding sub-image is utilized to segment crack from the crack saliency map. Then, the morphology operation is used to remove some isolated non-crack pixels and minimum spanning tree is applied to connect crack fragments. Finally, the accurate and complete crack can be located in the inspection image by computing crack skeleton.
(3)
(4)
(5)
Considering the orientation 𝑢𝜃 = (cos 𝜃, sin 𝜃)𝑇 , the basic steerable filter with 𝜃 ∈ [−𝜋∕2, 𝜋∕2] can be obtained by Eq. (6). And its detailed expression is shown in Eq. (7). ) ( ( ) 𝑒 𝑝0 , 𝜃; 𝜎 = 𝑢𝑇𝜃 𝐻 𝑝0 𝑢𝜃 (6) ) ( 𝑒 𝑝0 , 𝜃; 𝜎 = 𝑔𝑥𝑥 𝑐𝑜𝑠2 𝜃 + 𝑔𝑦𝑦 𝑠𝑖𝑛2 𝜃 + 𝑔𝑥𝑦 𝑠𝑖𝑛2𝜃
(7)
After the basic steerable filter e convolutes an image f(p), the filter response E at a position p0 can be calculated by Eq. (8). In addition, the scale size 𝜎 in the steerable basic filter can be fixed to adapt different width. As shown in Fig. 6(a), the first row shows three basic steerable filters with 𝜃 = 0, −𝜋∕4, 𝜋∕3, respectively. ( ( ( ) ) ) (8) 𝐸 𝑝0 , 𝜃; 𝜎 = 𝑒 𝑝0 , 𝜃; 𝜎 ∗ 𝑓 𝑝0
3.1. Crack saliency map generation by steerable evidence filtering The steerable filters derived from a linear combination of basic filter with arbitrary orientations [25]. In this study, the basic filter generates from Hessian Matrix H(p), which is a square matrix and describes the local curvature of the function. For a 2-D image f(p), at a position 𝑝0 = (𝑥, 𝑦), the hessian function can be obtained by Eq. (1). The sign ′∗ ′ represents a convolution operation. ( )) ( ( ) ( ) ( ) 𝑔𝑥𝑥 𝑝0 𝑔𝑥𝑦 𝑝0 ( ) ( ) ∗ 𝑓 𝑝0 𝐻 𝑝0 = (1) 𝑔𝑦𝑦 𝑝0 𝑔𝑥𝑦 𝑝0
However, the single basic steerable filter is incapable of detecting sharp bends, intensity variations, and complicated crack morphology. Thus, inspired by the local directional evidence filter [26], we design two additional oriented filters that include a certain offset in the angle and space distance of the detection point. Eqs. (9) and (10) are the corresponding two offset point p1 and p2 . Fig. 6(b) is the demonstration of two additional oriented filters. It shows a local search region around the detection point 𝑝0 (𝑥, 𝑦) = (1, 1), and the parameter settings are 𝑑 = 1, 𝜃 ∈ [−𝜋∕2, 𝜋∕2] and 𝜑 ∈ [−𝜋∕6, 𝜋∕6] with step of 𝜋/18. The yellow point
Eqs. (2)–(5) give the Gaussian kernel with variance 𝜎 and its corresponding second derivatives. ( ) ) ( 𝑥2 + 𝑦2 1 𝑔 𝑝0 ; 𝜎 = √ (2) 𝑒𝑥𝑝 − 2𝜎 2 2𝜋𝜎 25
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 6. (a) Example of the basic steerable filters and the SEFs with different orientations, respectively. (b) The local search region of the two additional oriented filters with offset point p1 and p2 . (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article)
represents the detection point p0 , and the red points represent the p1 , and p2 is displayed with green points, all of which appear as a circular region with the center of detection point p0 and the radius of d. Specially, the red points and green points partially overlap caused by the offset angle 𝜑 and the offset distance d, which ensure the detection of cracks at the entire circular region in the detection point. Thus, as shown in Eq. (11), the linear superposition of the basic steerable filter and two additional oriented filters forms the steerable evidence filter (SEF). As shown in Fig. 6(a), the second row shows three steerable evidence filters with different orientations. It indicates the SEF is consistent with the geometric characteristics of the crack fragments including line and curve, so the problems of crack bending, bifurcation and complicated geometry can be addressed. 𝑝1 = [𝑥 − 𝑑 cos (𝜃 + 𝜑), 𝑦 + 𝑑 sin (𝜃 + 𝜑)]
(9)
𝑝2 = [𝑥 + 𝑑 cos (𝜃 + 𝜑), 𝑦 − 𝑑 sin (𝜃 + 𝜑)]
(10)
( ( ) ( ) ) 𝑒∗ (𝑝; 𝜎, 𝜃, 𝜑) = 𝑒 𝑝0 ; 𝜎, 𝜃 + 𝑒 𝑝1 ; 𝜎, 𝜃 + 𝜑 + 𝑒 𝑝2 ; 𝜎, 𝜃 + 𝜑
(11)
paper, the threshold for segmenting crack from the crack saliency map is given by Eq. (14) 𝑇 (𝑥, 𝑦) = 𝜇𝑠 (𝑥, 𝑦) + 𝑘𝜎𝑠 (𝑥, 𝑦)
where 𝜇 s (x, y) and 𝜎 s (x, y) are the mean and standard deviation of each sliding sub-image of size N × N in S(x, y), and the size of mask is set 9 × 9 in our application. k is a predetermined constant. We select three EL images, and the corresponding segmentation results are given by Eq. (15). Although some background pixels are removed after the threshold process, there are still some crystal grains structures in the threshold segmentation result, so morphological operation is adopted to remove some small non-crack pixels and the final segmentation results are shown in Fig. 8. However, for these EL images with crystal grains interference, the segmented crack is disconnected near the crystal grains, which makes crack incomplete, as shown in Fig. 8(b) and (c). { 1, 𝑖𝑓 𝑆 (𝑥, 𝑦) > 𝑇 (𝑥, 𝑦) 𝐵 (𝑥, 𝑦) = (15) 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 To get a complete crack, minimum spanning tree (MST) based on Kruskal’s algorithm is employed. Minimum spanning tree is an important model in graph theory and it is used to solve the problem of minimal path cost [27]. As an illustration, Fig. 9(a) includes some nodes that have undirected connected graphs, and Fig. 9(b) is the complete tree by minimum spanning tree. In this process, minimum spanning tree can minimize the sum of the weights of corresponding edges and connect these nodes. After that, we use minimum spanning tree to connect crack fragments. Fig. 9(c) shows some crack fragments, which are caused by the crystal grains submerge the crack. The nodes of a1 and b1, c1 and d1 are the vertexes of crack fragments to be connected, respectively, and Fig. 9(d) shows the complete cracks after minimum spanning tree operation. In order to mark the crack defects, we compute the crack skeleton and locate them in the EL images. As shown in Fig. 9(e), the complete crack defects marked as red are accurately located.
In our application, we aim at finding the crack that lies in some uncertain position and orientation in an EL image. When convoluting an image with the SEF in Eq. (12), higher response magnitudes will be obtained in the crack region. To better highlight crack information, the maximum response magnitudes R∗ are calculate by Eq. (13), which form the crack saliency map S(x, y) for subsequent crack extraction. Fig. 7 shows three crack EL images and the corresponding saliency maps generated by the SEF. In addition, we focus on the different response of a defect-free patch and a crack patch. It shows the response magnitudes of defect-free region can be approximately treated as zero. In contrast, the crack region has higher response magnitudes, which makes crack salient and contributes to the further crack acquirement. 𝑅(𝑝; 𝜎, 𝜃, 𝜑) = 𝑒∗ (𝑝; 𝜎, 𝜃, 𝜑) ∗ 𝑓 (𝑝)
(12)
𝑅∗ (𝑝; 𝜎) = max 𝑅(𝑝; 𝜎, 𝜃, 𝜑)
(13)
𝜃,𝜑
(14)
4. Experimental results and discussion
3.2. Segmentation-based crack extraction from the saliency map
4.1. Parameter setting
In this section, we extract crack from the saliency map S(x, y) based on segmentation method. Owing to the random crystal grains, the defect-free region also presents certain response magnitudes in the saliency map. In order to extract the crack, we first apply a local threshold based on sliding sub-image to segment crack from the saliency map. Then, according to Tsai’s methods [10,17], they used a simple statistical control limit method to determine the threshold. Similarly, in this
In order to verify the performance of the proposed method, we collect the defective and defect-free EL images captured by the infrared camera on the production line in Tianjin Yingli new energy Co., Ltd. Here, 372 defective including different types’ cracks and 200 defect-free EL images are used to verify the performance of the SEF method. Although there are several parameters in the proposed method, not all of them play important roles in crack detection performance. In 26
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 7. Example of the crack saliency map. (a) Original crack EL images. (b) The saliency maps generated by the SEF. Fig. 8. Crack fragments obtained by local threshold in the saliency map of EL images. (a) Complete crack. (b) and (c) Crack fragments.
Fig. 9. Demonstration of minimum spanning tree and crack fragments connection. (a) Nodes to be connected. (b) Complete tree. (c) Crack fragments. (d) Connected crack. (e) Located crack.
27
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 10. Crack detection performance changes with different parameters values. (a)–(d) Four defective EL images. (a1)–(a3) Effect of scale size with 0.5, 1 and 2. (b1)–(b3) Effect of offset with 0.5, 1 and 2. (c1)–(c3) Effect of threshold k with 0.5, 1 and 1.5. (d) Effect of sub-image of size × N with 5 × 5, 9 × 9 and 17 × 17.
Table 1 Detail of the data set used in the experiment.
Types
Number of test images
Number of defective images used for optimal parameters selection
Pure crack Submerged crack Dendritic crack Break Dark crack Defect-free
113 170 17 49 23 200
20 20 10 10 10 20
our proposed scheme, four parameters are essential: SEF scale size 𝜎, offset d, threshold coefficient k and sub-image of size N × N. Fig. 10 shows the crack detection performance changes with different parameter values. The optimal parameter values in EL images of solar cells can be selected by assessing these results [17]. In this paper, in order to ensure the effectiveness of the selected optimal parameter values, we also apply some extensive experiments on selected a certain number of defective and defect-free EL images in Table 1. These extensive experiments have similar detection performance to Fig. 10. Moreover, the optimal detection results on selected EL images are counted under different parameter values, which are presented in Fig. 11. Concretely, the ordinate in Fig. 11 presents the number of EL images that can achieve the optimal detection performance of Fig. 10. For example, as shown in Fig. 11(a), the pink rectangle in pure crack type indicates that the crack can be accurately and completely extracted in all the 20 selected defective EL images when the scale size is set to 1. Particularly, all the 20 detection results can be achieved the optimal detection performance of Fig. 10(a2). Similarly, the blue rectangle in break type of Fig. 11(b) shows that altogether 7 defective EL images obtain the detection performance of Fig. 10(b2). In addition, all the detection results are achieved when the offset is set to 0.5. Other detection results of different parameters values in Fig. 11 are also obtained in this way. Fig. 11 presents the detection results according to different parameters values for the selected EL images. In the steerable evidence filtering procedure, we have to determine two parameters: the scale size 𝜎 and the offset parameter d. The parameter 𝜎 fits different widths of crack defects. As shown in Fig. 10(a1)–(a3), if 𝜎 is set too small, the crack cannot be well highlight and it is easy to be missed. Conversely, if 𝜎 is set too large, the crystal grains near the crack will be treated as crack and affect the detection accuracy. As shown in Fig. 11(a), for different types of defective and defect-free images, a better detection results can be obtained when the scale size 𝜎 is 1. Similarly, as shown in Fig. 10(b1)– (b3), the offset parameter d controls the locality of the steerable evidence filter. While a small value of d does not contribute enough, a large value will introduce false pixels that do not belong to the same crack
Size of each test image (pixels by pixels) 125 125 125 125 125 125
× × × × × ×
125 125 125 125 125 125
structure. According to Fig. 11(b), the optimal performance is achieved when d is 1. Besides, the threshold coefficient k will affect the detection results in the segmentation-based crack extraction process. As shown in Fig. 10(c1)–(c3), the small k gives a tight threshold and may identify the background pixels as crack pixels. However, large k gives a loose threshold and may miss some true defect pixels. As shown in Fig. 11(c), although the same detection results can be obtained for dendritic-type crack images and defect-free images when k is 1 or 1.5, the k is set to 1 for the whole types of EL images. Additionally, the size N × N of the subimage in the local threshold process plays a certain role in the final crack segmentation results. As shown in Fig. 10(d1), if the value of N is too small, the mean and standard deviation are calculated in a small image region. It includes more local details such as crystal grains, making it difficult to distinguish crack from these crystal grains. Therefore, crack is easily removed in the subsequent processing. Conversely, as shown in Fig. 10(d3), if the value of N is too large, some non-crack background pixels are easily to be identified as crack fragments. Figs. 10(d2) and 11(d) show the optimal performance is achievement when the size of sub-image is 9 × 9.
4.2. Robustness analysis in brightness levels In this study, we also test some dim defective images that present low contrast between crack and background and defect-free images, and the proposed method still achieve satisfactory performance. Fig. 12(d1) is a dim defect-free EL image, and Fig. 12(a1)–(c1) are three typical low-efficiency solar cells that show large dark areas, causing the weak differences between crack defect and background in the corresponding EL images. The corresponding crack saliency maps are shown in Fig. 12(a2)–(d2), in which crack defects become salient. The detection result of Fig. 12(d3) shows that no crack is detected in the defect-free EL image. The crack defects still can be complete and accurately located by red curves in the dim defective EL images simultaneously. Therefore, the proposed method is robust to different brightness levels. 28
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 11. Optimal parameter values selection. (a) Effect of scale size. (b) Effect of offset d. (c) Effect of threshold k. (d) Effect of sub-image of size N × N. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article)
4.3. Performance evaluation
gradient in EL images. Moreover, the crack detection results are intermittent, such as in Fig. 13(a2) and (e2). Specially, this method is less adept at detecting the low contrast crack in the dim EL images, which is shown in Fig. 13(f2). The Chiou’s method uses a local threshold-based algorithm to segment crack. It mainly partitions an image into several sub-images and calculates each sub-image’s mean and standard deviation. The detection results are shown in Fig. 13(a3)–(g3), and the crystal grains are detected as crack due to only take intensity information into account. In addition, the crack defect cannot be detected in the dim EL images, such as in Fig. 13(f3). The Anwar’s method applies an improved anisotropic diffusion filtering with intensity-based diffusion coefficient, which can enhance the crack pixels with low gray level and high gradient. In addition, shape analysis is performed to distinguish between crack and other arbitrary patterns. To get optimal detection performance, parameter b is set to 1, and the iteration t is set to 4 in this paper. The corresponding detection results are shown in Fig. 13(a4)–(g4). Concretely, Fig. 13(b4) shows some random crystal grains also appear as low intensity and high gradient features. These crystal grains show crack-shaped in the segmentation results, which are easily to be identified as crack. In addition, for low contrast defective image such as Fig. 13(f4), this crack detection method achieves poor performance due to the weak intensity and gradient differences between crack and surrounding background. In contrast, for defect-free image in Fig. 13(g4), this method has good robustness.
4.3.1. Qualitative evaluation In this paper, our SEF method is compared with the Tsai’s method [10], Chiou’s method [11], Anwar’s method [21], and the basic steerable filter (BSF) method [25] on the given image data set. The performance of four comparison methods may be not fully effective relative to the original work due to the changes in the data set. However, we have adjusted some key parameters to make these methods more suitable for the current data set and ensure a relatively high detection rate. As an illustration, Fig. 13 shows representative six defective EL images, one defect-free EL image and the corresponding detection results. The first row shows the original images and the corresponding detection results are given from the second column to sixth column, respectively. The seventh column is the manually labeled ground truth images of crack defect. The Tsai’s method assumes that the crack defect presents the features of low gray level and high gradient. In that case, the anisotropic diffusion model will generate high diffusion coefficients to smooth crack defect and preserve the gray level of defect-free background, simultaneously. In order to get better detection performance, we set the diffusion iterations 𝑇 = 3, regularization value 𝑘 = 4, and control constant 𝑐 = 2, and the detection results are shown in Fig. 13(a2)–(g2). Experimental results show that it mistakes some crystal grains as crack defect because the random crystal grains also exhibit features of low gray level and high 29
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 12. Demonstrative luminance robustness of the SEF method.
The BSF method uses the single basic steerable filter to process the EL images. Fig. 13(a5) shows that the BSF is less adept at solving intensity variations, making the crack incomplete. Moreover, the BSF method cannot well address the bifurcations due to the lack of consideration of local neighborhood information. As shown in Fig. 13(d5), the crack in the bifurcation is missed. However, for the dim EL images, this method can achieve a better result, such as in Fig. 13(f5). The detection results based on the SEF method are shown in Fig. 13(a6)–(g6). Although many crystal grains around the crack, the proposed method can detect complete crack and locate it in the inspection image. Moreover, all the false detection of crystal grains by the Tsai’s method, Chiou’s method and Anwar’s method are well addressed. Furthermore, the SEF method can well detect crack in the dim EL images and it is robust to defect-free image, such as Fig. 13(f6)–(g6). Overall, it achieves much better performance than other three methods.
Eq. (18) is the F-measure index, which can comprehensively evaluate the performance of the crack detection algorithm. The higher the value, the more effective the detection algorithm is. 𝐹 − measure =
𝐿 𝐿𝑔
(16)
𝑐𝑟𝑡 =
𝐿 𝐿𝑡
(17)
(18)
For five different types of defective EL images, the quantitative evaluation based on the statistic results is shown in Table 2. The results obtained by the Tsai’s, Chiou’s, Anwar’s, BSF method, and the SEF method are shown, respectively. For example, the first row of Table 2 shows the detection results obtained on the pure-type crack images using four different methods. The cpt, crt, and F-measure are given in different columns for each method individually. The last row shows the overall performance of different methods. From Table 2, it can be observed that the Tsai’s method has a lower cpt than the other methods owing to the similarities of gray levels and gradients between crack and crystal grains. Especially, the Tsai’s method performs badly for dark-type crack in dim EL images. Moreover, for most of the break defects, there are scarcely crystal grains interferences in the background, so the Tsai’s method achieves a better performance. In contrast, for the other crack defect types, there are randomly distributed crystal grains in the background, which leads to lower detection rate. The Chiou’s method is still unable to detect dark-type cracks because of weak difference between crack and surrounding background. Although most of crack pixels can be identified, many crystal grains are mistaken as crack and the detection results are inaccurate. The Anwar’s method cannot effectively detect dark-type cracks in low contrast images. The BSF method mainly shows poor performance on intensity variations and bifurcations. In contrast, the SEF method can get the better detection rate than the other three methods. As the more intuitive description, a graphical representation of Fmeasure based on the above five methods is shown in Fig. 14. From the experimental results in Fig. 14, it can be seen that the proposed SEF method gives the better detection results than other four methods. In particular, for break and dark-type cracks, it has a high robustness and
4.3.2. Accuracy evaluation For quantitatively estimating the performance of the proposed SEF method for multicrystalline solar cells, we apply three evaluation indices for the accuracy evaluation: completeness (cpt), correctness (crt) and Fmeasure, which are used to evaluate the crack detection performance in EL images of solar cells [21] and pavement surface [28]. Specifically, the cpt index and crt index explain the completeness and effectiveness of crack detection algorithm, respectively. The cpt and crt are calculated as shown in Eqs. (16) and (17). 𝑐𝑝𝑡 =
2 × 𝑐𝑝𝑡 × 𝑐𝑟𝑡 𝑐𝑝𝑡 + 𝑐𝑟𝑡
where Lg is the number of crack pixels in the corresponding ground truth image obtained from manual marking. L is the number of pixels in the crack detection result which matches the ground truth crack pixels. Lt is the total number of extracted pixels in the detection result. 30
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Fig. 13. Comparison of the proposed method with previous methods. (a1)–(g1) Six defective EL images and one defect-free image. (a2)–(g2) Detection results of the Tsai’s method. (a3)–(g3) Detection results of the Chiou’s method. (a4)–(g4) Detection results of the Anwar’s method. (a5)–(g5) Detection results of BSF method. (a5)–(g5) Detection results of proposed method. (a7)–(g7) Crack ground truth. All the detective regions are labeled as red.
achieves the best detection performance. Overall, the performance of the proposed SEF method is better than that of the other methods. In addition, we also qualitatively evaluate the correctness of different methods for 200 defect-free images. The detection results are summarized in Table 3. It shows the proposed method is more robust to defect-free images. In terms of computational efficiency, as shown in Table 4, for per test image with the size 125 × 125 pixels, the SEF method takes more
time than the Tsai’s method, Chiou’s method, Anwar’s method and BSF method. However, considering the crack detection results, current detection time is also tolerable. 4.4. Further analysis and discussion In steerable evidence filtering process, we can effectively obtain the crack saliency map, which includes the candidate crack pixels. 31
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
Table 2 Experimental performance on 5 different types of crack defects using different methods. Types of cracks
Tsai’s method
Pure crack Submerged crack Dendritic crack Break Dark-type crack Overall performance
Chiou’s method
Anwar’s method
BSF method
SEF method
cpt
crt
Fm
cpt
crt
Fm
cpt
crt
Fm
cpt
crt
Fm
cpt
crt
Fm
37.6 64.5 53.3 80.9 38.5 55.0
63.6 84.5 80.0 82.9 33.3 68.9
47.3 73.2 64.0 81.9 35.7 60.4
54.1 79.6 58.8 94.7 31.8 63.8
97.0 94.2 100 76.6 87.5 91.1
69.5 86.3 74.1 84.7 46.4 72.2
89.1 79.3 78.2 89.2 56.7 78.5
90.2 85.4 80.9 88.5 59.4 80.9
89.6 82.2 79.5 88.8 58.0 79.6
89.3 85.9 78.8 93.6 94.5 88.4
90.4 93.0 82.5 100 95.4 92.3
89.8 89.3 80.6 96.7 94.9 90.3
95.0 88.3 92.3 97.2 96.7 93.9
91.1 94.6 89.5 100 100 95.0
93.0 91.3 90.9 98.6 98.3 94.4
Fig. 14. Variation of F-measure by different methods.
Table 3 The qualitative evaluation for defect-free images. defect-free images
Tsai’s method
200
defect-free 157
defective 43
Chiou’s method
Anwar’s method
BSF method
defect-free 139
defect-free 193
defect-free 191
defective 61
Table 4 Comparison of computational time. Methods
Times (ms)
Tsai’s method Chiou’s method Anwar’s method BSF method SEF method
72 64 83 79 86
defective 7
SEF method defective 9
defect-free 198
defective 2
(2) Crystal grains with similar intensity to cracks are mistaken as defects in Tsai’s, chiou’s and Anwar’s methods. The SEF method avoids this problem. (3) In other four methods, the detected cracks are incomplete due to inconsistent intensity and bifurcation. However, this problem can be solved by steerable evidence filter and minimum spanning tree to acquire complete crack. In addition, the SEF method has some deficiencies that need to be further exploration. First, some parameters such as the scale size 𝜎 and the offset parameter d, are experimentally determined by a certain number of test images, so it may not be the optimal value for all large of amounts of images. So, if we can exploit the correlation of these parameters to achieve the goal of adaptive adjustment, then the detection performance will be further improved. Second, after the local threshold procedure, some small fragments are similar to crack fragments. Segmented crack may miss a portion when we use the morphological operation to remove some non-crack pixels. Thus, the detection performance will be affected when some curvilinear crystal grains are in the background.
During the crack extraction process, we analyze the crack saliency map by local threshold and minimum spanning tree to acquire the complete crack. Comparing with Tsai’s and Chiou’s methods, which just use the intensity information to identify crack from the background and mistake many crystal grains as crack pixels. Anwar’s method uses improved anisotropic diffusion and segmentation technique to obtain crack defect. This method cannot remove crack-shaped crystal grains that appear low intensity and high gradient features. Especially, it is difficulty for these three methods to detect crack in the dim EL images, which show small intensity differences between crack and surrounding background. In addition, the BSF method fails to address intensity variations and bifurcations. Thus, the SEF method has three advantages over these four methods.
5. Conclusion In this paper, an automatic crack detection system for multicrystalline solar cells is presented which utilize the electroluminescence (EL) imaging technique to capture crack defect information.
(1) The crack defects in the dim EL images, which Tsai’s, chiou’s and Anwar’s methods cannot detect are well addressed by the SEF method, as this dark-type crack is not salient in the dim background. In addition, for defect-free images, the SEF method is more robust than other methods.
(1) A novel steerable evidence filter is developed to enhance the contrast between crack and background and provide evidence for the presence of crack defect. Then, local threshold and minimum 32
H. Chen, H. Zhao and D. Han et al.
Optics and Lasers in Engineering 118 (2019) 22–33
spanning tree are employed to extract complete crack. Finally, crack defect can be accurately located in the inspection image. (2) Compared with previous methods, the proposed SEF method is more robust to the heterogeneously textured background interferences, crack types and image brightness levels change. Specially, this method can well solve low contrast and complex cracks with sharp bends and bifurcations. Moreover, considering the accuracy, robustness and processing time of crack detection, the proposed SEF method can achieve satisfactory detection results and meet the practical online detection requirement.
[19] Zhang W, Zhang Z, Qi D, et al. Automatic crack detection and classification method for subway tunnel safety monitoring. Sensors 2014;14(10):19307–28. [20] Prasanna P, Dana KJ, Gucunski N, et al. Automated crack detection on concrete bridges. IEEE Trans Autom Sci Eng 2016;13(2):591–9. [21] Anwar SA, Abdullah MZ. Micro-crack detection of multicrystalline solar cells featuring an improved anisotropic diffusion filter and image segmentation technique. EURASIP J Image Video Process 2014;2014(1):15. [22] Demant M, et al. Microcracks in silicon wafers I: inline detection and implications of crack morphology on wafer strength. IEEE J Photovoltaics 2016;6(1):126–35. [23] Zhang L, Yang F, Zhang YD, et al. Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing, IEEE; 3708–3712. [24] Chen FC, Jahanshahi MR. NB-CNN: deep learning-based crack detection using convolutional neural network and naïve bayes data fusion. IEEE Trans Indust Electron 2018;65(5):4392–400. [25] Freeman WT, Adelson EH. The design and use of steerable filters. IEEE Trans Pattern Anal Mach Intell 1991;13(9):891–906. [26] Mukherjee S, Acton ST. Oriented filters for vessel contrast enhancement with local directional evidence. In: 2015 IEEE 12th International Symposium on Biomedical Imaging. IEEE; 2015. p. 503–6. [27] Graham RL, Hell P. On the history of the minimum spanning tree problem. Ann History Comput 1985;7(1):43–57. [28] Li Q, Zou Q, Zhang D, et al. FoSA: F∗ seed-growing approach for crack-line detection from pavement images. Image Vision Comput 2011;29(12):861–72.
Acknowledgement This work was supported in part by National Natural Science Foundation (NNSF) of China under grant 61873315, 61403119, Natural Science Foundation of Hebei Province under grant F2018202078, Tianjin Science and Technology Commissioner Project under grant 18JCTPJC55200, Science and Technology Program of Hebei Province under grant 17211804D, Hebei Province Outstanding Youth Science Fund (F2017202062), and Young Talents Project in Hebei Province under Grant 210003.
Haiyong Chen received the M.S. degree from the Harbin University of Science and Technology, Harbin, China, in 2005, and the Ph.D. degree from the Institute of Automation, CAS (Chinese Academy of Sciences), Beijing, China, in 2008. He is currently a Professor with the School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. His research interests include image processing, robot vision, and pattern recognition.
Supplementary materials Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.optlaseng.2019.01.016. References [1] Kabir E, Kumar P, Kumar S, et al. Solar energy: potential and future prospects. Renewable Sustainable Energy Rev 2018;82:894–900. [2] Abdelhamid M, Singh R, Omar M. Review of microcrack detection techniques for silicon solar cells. IEEE J Photovoltaics 2014;4(1):514–24. [3] Dhimish M, Holmes V, Mehrdadi B, et al. The impact of cracks on photovoltaic power performance. J Sci 2017;2(2):199–209. [4] Israil M, Anwar SA, Abdullah MZ. Automatic detection of micro-crack in solar wafers and cells: a review. Trans Inst Meas Control 2013;35(5):606–18. [5] Frazao M, Silva JA, Lobato K, et al. Electroluminescence of silicon solar cells using a consumer grade digital camera. Measurement 2017;99:7–12. [6] Nhat-Duc H. Detection of surface crack in building structures using image processing technique with an improved otsu method for image thresholding. Adv Civil Eng 2018;2018:1–10. [7] Kamaliardakani M, Sun L, Ardakani MK. Sealed-crack detection algorithm using heuristic thresholding approach. J Comput Civil Eng 2014;30(1):04014110. [8] Sun L, Kamaliardakani M, Zhang Y. Weighted neighborhood pixels segmentation method for automated detection of cracks on pavement surface images. J Comput Civil Eng 2015;30(1):04015021. [9] Cinar AF, Barhli SM, Hollis D, et al. An autonomous surface discontinuity detection and quantification method by digital image correlation and phase congruency. Opt Lasers Eng 2017;96:94–106. [10] Tsai DM, Chang CC, Chao SM. Micro-crack inspection in heterogeneously textured solar wafers using anisotropic diffusion. Image Vision Comput 2010;28(3):491–501. [11] Chiou YC, Liu JZ, Liang YT. Micro crack detection of multi-crystalline silicon solar wafer using machine vision techniques. Sens Rev 2011;31(2):154–65. [12] Ko SS, Liu CS, Lin YC. Optical inspection system with tunable exposure unit for micro-crack detection in solar wafers. Opt Int J Light Electron Opt 2013;124(19):4030–5. [13] Xu K, Xu Y, Zhou P, et al. Application of RNAMlet to surface defect identification of steels. Opt Lasers Eng 2018;105:110–17. [14] Khan H A, Salman M, Hussain S, Khurshid K. Automation of optimized Gabor filter parameter selection for road cracks detection. Int J Adv comput Sci Appl 2016;7(3):269–75. [15] Zalama E, Gómez-García-Bermejo J, Medina R, et al. Road crack detection using visual features extracted by Gabor filters. Computer-aided Civil Infrastruct Eng 2014;29(5):342–58. [16] Medina R, Llamas J, Gómez-García-Bermejo J, et al. Crack detection in concrete tunnels using a Gabor filter invariant to rotation. Sensors 2017;17(7):1670. [17] Tsai DM, Wu SC, Li WC. Defect detection of solar cells in electroluminescence images using Fourier image reconstruction. Sol Energy Mater Sol Cells 2012;99:250–62. [18] Ouma YO, Hahn M. Wavelet-morphology based detection of incipient linear cracks in asphalt pavements from RGB camera imagery and classification using circular Radon transform. Adv Eng Inf 2016;30(3):481–99.
Huifang Zhao received the B.S. degree from the Hebei University of Science and Technology, Shijiazhuang, China, in 2016. She is currently pursuing the M.S. degree in automation, School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. Her current research interests include computer vision and pattern recognition.
Da Han received the B.S. degree from the Hebei University of Science and Technology, Shijiazhuang, China, in 2016. He is currently pursuing the M.S. degree in automation, School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. His current research interests include image processing and machine learning.
Kun Liu received the M.S. degree from the Harbin Institute of Technology, Harbin, China, in 2003, and the Ph.D. degree in automation from the Tsinghua University, Beijing, China, in 2009. She is currently an Associate Professor with the School of Artificial Intelligence and Data Science, Hebei University of Technology, Tianjin, China. Her research interests include image processing, computer vision, and pattern recognition.
33