- Email: [email protected]
Defect detection in X-ray images using fuzzy reasoning
Image and Vision Computing 19 (2001) 261±269
www.elsevier.com/locate/imavis
Defect detection in X-ray images using fuzzy reasoning V. Lashkia* Department of Informatics and Computer Engineering, Okayama University of Science, 1-1 Ridai-cho, Okayama, 700-0005, Japan Received 19 July 1999; revised 1 August 2000; accepted 4 September 2000
Abstract Since X-ray images contain sensory noise, and objects in X-ray images are often distorted by various effects such as uneven lighting, classical image processing techniques and methods based on ordinary crisp set theory are poor at detecting small low contrast objects. In this paper, we propose a more effective method based on fuzzy theory. With the proposed algorithm, images are ®ltered by applying fuzzy reasoning using local image characteristics. The proposed algorithm was applied to detect internal weld defects from radiographic ®lms, which are taken from steel butt weld parts. Results show success in detecting defects at a similar level to human vision. A comparison between a visual and an automatic evaluation demonstrates the ef®ciency of this method. q 2001 Elsevier Science B.V. All rights reserved. Keywords: Non-destructive testing; Visual inspection; Defect detection; Fuzzy reasoning; X-ray radiography; Weld defect
1. Introduction The problem of determining what is and what is not an object in X-ray images is compounded by the fact that objects are often distorted. The objects are often seen to contain a number of regions having fairly distinct gray levels superimposed on the background level. Furthermore, X-ray images frequently contain data with object-like characteristics, such as edges with a high gradient and noise. Images are also corrupted by non-uniform illumination. In the image, most objects can be seen as a zone whose brightness is different from the surrounding area. A human can easily ®nd such a zone or boundary even when the image is corrupted by non-uniform illumination, contains noise and has low contrast. However, most of the current image processing algorithms encounter dif®culties in detecting such zones. A method that can successfully detect objects in X-ray images has many applications in medicine (detection of distortions in mammograms, etc. [2]) and in non-destructive testing of industrial products (detection of defects in welding seams [3,8±11,14,15,17] and composite materials [10]). In this paper, we propose a new method for small low contrast object detection based on fuzzy theory and apply it to the detection of defects in X-ray images of steel butt weld parts. Digital image processing in industrial radiography is used primarily for preprocessing and restoration of X-ray images * Tel.: 181-86-256-9592; fax: 181-86-255-3611. E-mail address: [email protected] (V. Lashkia).
[3,14]. Detection and classi®cation of weld defects is still done by a human observer. The non-destructive testing of welding seams involves a series of processes from sticking a ®lm-pack piece by piece along the run of the welding seam, exposing the pieces of ®lm with X-rays, to developing them to be identi®ed by humans. Because many pieces of X-ray ®lm need to be interpreted in a short time and some of the defects like transverse cracks are dif®cult to see, an automatic test approach is highly desirable. Current processing algorithms for defect detection in Xray images suffer from a number of problems. Algorithms that use smoothing operators [11,14,17] have dif®culties in detecting small objects such as cracks, which appear as a subtle change of shade in the image. When these operators are used the small low contrast objects become blurred or can completely disappear. Algorithms based on differential image calculation (between the original image and background image without defects) [3,11] are also unable to deal with small low contrast defects, as the processing is strongly affected by the size and contrast of a defect. Edge detecting techniques [10,14] are found to be effective only when there is signi®cant contrast. Attempts to detect edges of small low contrast defects makes the method sensitive to noise. Conventional image segmentation methods such as adaptive thresholding [10,15] or iterated conditional modes [10] are unable to properly segment non-uniformly illuminated images. To overcome non-uniformly illuminated images. To overcome non-uniform illumination problems a new adaptive threshold method based on Yanowitz and Bruckstein's algorithm with a Canny edge detector was
0262-8856/01/$ - see front matter q 2001 Elsevier Science B.V. All rights reserved. PII: S 0262-885 6(00)00075-5
262
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
Fig. 1. Incomplete penetration (image is magni®ed).
proposed in Ref. [10]. This method uses edge points to interpolate the threshold surface, and works well for relatively large defects but does not differentiate between small defects and noise. In Refs. [8,9], an adaptive model-based defect detection method RP was introduced. RP detects objects by ®tting to them a ªconcentric netº model, which is built from radiated rays and concentric curves. By evaluating local statistical parameters, such as spatial contrast and variance, and employing commonsense reasoning rules, which may resemble the heuristic rules employed by humans, RP adjusts its frames. Also, unlike many other recognition schemes it does not rely on some form of pre-normalization of input images, and can handle distorted low contrast images as well as non-distorted high contrast images. Although this method shows good experimental results, it requires many parameters and threshold values that must be set empirically. In an attempt to improve the performance of the RP method we introduce a new approach which implements fuzzy reasoning using a neuro-fuzzy system, and which makes the thresholding process smoother and more ¯exible. Since small low contrast defects include fatal defects (cracks), the performance of the defect detection method needs to be at the same level as that of ®eld inspectors. To perceive objects in imperfect images, it appears that humans are applying heuristic algorithms to understand such images [13]. We believe that it is possible to characterize an equivalent process systematically. We propose that fuzzy reasoning is a suitable framework for expressing heuristic processes applied to imperfect data. Fuzzy reasoning, as outlined by Zadeh [19], with the power to model and respond usefully to approximate situations, is ideally suited to this problem. The proposed method acknowledges the existence of distorted low contrast defects as well as non-distorted high contrast defects, and elaborates a linguistic framework for characterizing these imperfections. These characteristics are
approximated from local statistical parameters (such as spatial contrast and spatial variance), and fuzzy rules based on heuristics are introduced to evaluate these parameters. Analogous approaches have been introduced in Refs. [13,18], where fuzzy reasoning based on local statistical parameters was successfully applied to edge detection, achieving better performance than conventional crisp methods. Here, we report the ®rst attempt to detect low contrast defects in X-ray ®lms with a fuzzy reasoning approach. The major advantages of the proposed method are as follows: (1) it is based on commonsense fuzzy reasoning rules, which offer an intrinsic understanding of the classi®cation logic; (2) the parameters and threshold values are not selected empirically as in many conventional methods, but are obtained by the training process; (3) the method is simple (only values of spatial contrast and spatial variance are used to determine the existence of a defect); (4) it is ¯exible (the method works for distorted low contrast images as well as non-distorted high contrast images); and (5) it is stable (the proposed method was tested on more than 100 images, and results show success in detecting defects at a similar level to human vision). This paper is organized as follows. In Section 2, we describe the principles behind the proposed method. In Section 3, we apply the proposed method to the problem of defect detection in X-ray images of steel butt weld parts. In Section 4, we discuss experimental results. 2. Proposed fuzzy reasoning In X-ray images, small objects can often be seen as a zone with low contrast. Because noise in X-ray images has a spotty contrast, small objects cannot be detected by methods based only on contrast characteristics. On the other hand, although noise can be seen as a zone with high spatial variance, methods based on this feature are ineffective because objects exhibit the same features due to distortion and uneven lighting. However, the combination of contrast and variance could be used to distinguish objects from noise. This is because an object is different to noise in that it has low spatial variance when the contrast is low (and therefore appears as a zone whose brightness is different from the surrounding area). In Fig. 1, we show an example of a defect in a steel butt weld part. The defect has low contrast and low spatial variance, as shown in Fig. 2. If the contrast increases, an object can have intensity variation (Figs. 3 and 4), and the higher the contrast the easier the object is to detect. These ideas are expressed as a 3D surface in Fig. 5. For a ®xed contrast value, the lower the variance the more con®dent we can be that the observed area is an object. For a ®xed variance value, the higher the contrast the more con®dent we can be that the observed area is an object. The proposed method of fuzzy reasoning is based on Fig. 5 and fuzzy rules of the proposed defect detection system are formed according to it.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
Fig. 2. 3D representation of the image from Fig. 1.
One more parameter which is useful for object detection in X-ray images is the distance between two regions characterizing the contrast parameter. Our investigations of noise areas in X-ray images show that contrast regions separated by short distances indicate a strong possibility of noise. On the other hand, for medium distances the contrast of noise area becomes low, and for long distances the distinction between the object and noise is confusing. In Fig. 6 we show an area of an X-ray image of a steel butt weld part containing no defects. As seen from Fig. 6, there is an intensity variation in the small region that surrounds particular isolated noise, although the difference between the intensity values of the isolated noise and an area that
263
Fig. 4. 3D representation of the image from Fig. 3.
is relatively far from it is, in general small, because almost all areas contain some noise pixels. This suggests that it is preferable to have higher con®dence of an object for medium distances than for short distances. The proposed method evaluates the object fuzzy membership value for each pixel, based on the local image characteristics described above. Pixels having high membership are detected as regions belonging to defects and then tracking and region growing methods are applied to detect the exact position and shape of the defects. To calculate a fuzzy variable we employ the simpli®ed fuzzy reasoning method proposed in Ref. [7]. The simpli®ed fuzzy reasoning method induces a neuro-fuzzy system, which was investigated in great detail in Refs. [4,6,12,16]. When the input of neuro-fuzzy system is de®ned as x1 ; ¼; xm and the output is y, the ith rule of the simpli®ed Membership for object
0 Contrast 0 Fig. 3. Slag inclusion (image is magni®ed).
Variance
Fig. 5. Control surface of the proposed fuzzy reasoning.
0
264
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
Fig. 7. A de®nition of a spatial contrast and variance.
average or a medium value of Fu : The value X Fu i; j F i; j
1
u
Fig. 6. Noise area (image is magni®ed).
fuzzy reasoning is given by the following expression: if x1 is Ai1 ¼ and xm is Aim then y is wi; where i i 1; ¼; n is the rule number, Ai1 ; ¼; Aim are membership functions, and wi is the real value of the consequent part. The output y of the neuro-fuzzy system is calculated as outlined below n X
y
wi mi
i1 n X i1
;
mi Ai1 x1 ¼Aim xm :
mi
Several methods for tuning parameters of membership functions of a neuro-fuzzy system were proposed. In this research we employ the method of steepest descent described in Refs. [5,7]. The proposed fuzzy reasoning approach can be implemented in various ways. Below, we present one of the possible implementations. The spatial contrast Cu i; j can be de®ned as a difference in the average values of pixels of regions E and D, see Fig. 7. The spatial variance Vu i; j can be de®ned as the standard variance of the region D. The size of regions D and E depends on the minimal size of the defect which we want to detect. Denote by Fu i; j the membership for an object calculated based on Cu ; Vu ; and n using a neuro-fuzzy system. We call Fu the object u -directional membership value. We determine if the i; j pixel is a part of the object or background, based on the values Fu i; j: A classi®er based approach (for example neural network) can be employed for this purpose. Another simple way to handle the values Fu is to employ a thresholding method. The threshold value can be chosen in various ways depending on the objects we are looking for. One possible way, which takes into account distorted and low contrast objects, is as follows. Suppose that wavr is an
we call the overall object membership value of the pixel i; j: We choose a threshold value X wavr ; 2 tr u
and may assume i; j is an object pixel if F i; j $ tr: We describe a more detailed implementation of the proposed method in Section 3. The important part of the proposed method is the training procedure. We train a neuro-fuzzy system using a training set, which is formed from small low contrast objects and noise. In this paper, we employ triangular and trapezoidal membership functions and use the following fuzzy set values: very small (VS), small (S), medium (M), and large (L). For a variable x we de®ne the fuzzy set VS as an area to the left of the minimal value of x from the object training samples. We also de®ne the fuzzy set L as an area to the right of the maximal value of x from the object training Weld boundaries
Pencil mark
Crack
512pixels
Fig. 8. A sample X-ray image of a welding seam.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
265
Grey value
Radiographic Film
1
crack
Pixels 512
ImageSampling and Gray-Level Quantization
Fig. 9. Plot of gray values along the defect.
samples. The area between VS and L is represented by fuzzy sets S and M.
FuzzyReasoning
Fuzzy Reasoning
(Vertical)
(Horizontal)
+
3. Application to defect detection DefectIsolation
The proposed method was applied to the problem of detecting internal weld defects from radiographic ®lms taken from steel butt weld zones. All radiographs are digitized by a TV-camera and stored in the computer. The resolution is 512 £ 480 pixels with 256 gray levels. Fig. 8 shows a digitized sample image of a transverse crack. The bright region in Fig. 8 is the weld zone. In Fig. 9, a plot of the gray values along the line of the crack is shown. The variation of the gray value of noise and the defect area is almost the same. Some of defects (such as transverse cracks) are dif®cult to ®nd even by human vision. Table 1 shows the names and visual features of the main weld defects. Various shapes of weld defects were found but we can roughly classify them into three classes. These are round-like defects (blow hole, slag inclusion, incomplete penetration), longitudinal defects (lack of fusion, incomplete penetration, crack) and transverse defects (cracks). Taking into account the possible shapes of defects it is reasonable to ®lter images by applying fuzzy reasoning in only the horizontal and vertical directions. In horizontal ®ltering we concentrate on the detection of longitudinal defects and in vertical ®ltering we concentrate on the detection of transverse defects. Round-like defects would be detected in both directions. Fig. 10 shows a ¯ow chart of the automatic detection of weld defects. The algorithm consists of the following parts: ² In the ®rst step, a region of interest on the weld radioTable 1 Names and visual features of the main weld defects Defect
Visual representation
Blow hole Slag inclusion Lack of fusion
Roundish zone having high contrast Large spread zone Long and narrow zone located between the center and boundary of weld zone Long and narrow zone located in the center of weld zone Thread-like zone with low contrast
Incomplete penetration Crack
Result
Fig. 10. Flow chart for the automatic detection of weld defects.
graph is digitized by a TV-camera and stored in the computer. ² The main part of the algorithm is the defect detection procedure. By applying fuzzy reasoning in vertical and horizontal directions the positions of defect kernels are detected. ² Tracking and region growing methods are applied to detect the exact position and shape of the defects. The result of the image analysis procedure is a binary image, where the background is white and defect indications are black.
3.1. Vertical ®ltering For the vertical processing, we use spatial contrast C and spatial variance V to evaluate the defect fuzzy membership value. The spatial contrast is determined by the characteristics of two regions one on either side of the center i0 ; j0 of the vertical ®lter shown in Fig. 11. To ignore noise effects Gradient direction En n (i 0, j 0) D (i, j) Testing direction
Fig. 11. Vertical ®lter.
266
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
we consider the average value of pixels in each region. By A En we denote the average value of the region En which is located n pixels away from the center in the direction of the gradient. By A D we denote the average value of the region D which is located in the direction opposite to the gradient (we call this the testing direction, because defect pixels are assumed to be in this direction) and centered at (i, j). In vertical ®ltering only up and down directions of the gradient at the pixel (i0, i0) are considered. For the (i, j) pixel we de®ne the spatial contrast as C i; j maxn A En 2 A D and the spatial variance V i; j as the standard variance of the region D. Examining noise pixels in sample images, we found that the spatial contrast values of noise pixels are concentrated in S and M areas of the defect pixel contrast values. Therefore, to achieve a high defect detection performance we choose narrower fuzzy set values and ®ner partitions for the S and M fuzzy values of variables C and V. The S and M fuzzy sets were divided into S1, S2, S3 and M1, M2, M3 fuzzy sets, respectively. According to Fig. 5 we set the inference rules of the simpli®ed fuzzy reasoning, as shown in Table 2, and use it to calculate the defect membership value. The values 1, 2, and 3 in Table 2 express the low, medium, and high con®dence accordingly. When spatial variance is low and spatial contrast is large, medium and small, a pixel is considered as a part of a defect with the membership value 3, 2 and 1, respectively as shown in Table 2. The defect membership value decreases when the spatial variance increases. The choice of defect membership values is made by taking into account distorted images and will be explained below. For a pixel (i, j) the proposed method evaluates the defect membership value based on rules shown in Table 2. Denote f1 i; j as the defect membership value calculated for (i, j) when the test direction is down. Denote f2 i; j as the defect membership value calculated for (i, j) when the test direction is up. When the ®lter across an image is moved, each pixel defect membership value can be calculated several times. For each pixel we calculate the down directional membership value F1 (i.e. Fu , where u p=2) as the maximum of membership values calculated for a pixel when the test direction was down, F1 i; j max f1 i; j: For each pixel we calculate the up directional membership value F2 (i.e. Fu , where u 2p=2) as the maximum of membership values calculated over a pixel when the test direction was up, F1 i; j max f2 i; j: According to Eq. (1), the overall defect membership value F of a pixel (i, j) is determined as a sum of its directional membership values F i; j F1 i; j 1 F2 i; j: The reason for using the maximum function in calculating F1 and F2 is to give almost the same defect membership value for pixels of small or low contrast defects as pixels of large or high contrast defects. The use of summation instead of maximum value makes thresholding (deciding pixels are part of a defect) complicated. Let wavr 2; which corresponds to the medium value of Fu . According to Eq. (2), a pixel is considered a part of a
Table 2 Fuzzy rules for vertical ®ltering C
V
VS S1 S2 S3 M1 M2 M3 L
VS
S1
S2
S3
M1
M2
M3
L
0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0
1 1 0 0 0 0 0 0
1 1 1 0 0 0 0 0
2 2 2 2 1 0 0 0
2 2 2 2 2 1 0 0
2 2 2 2 2 2 1 0
3 3 3 3 3 3 2 1
defect if and only if the overall membership P value of the pixel is greater than or equal to tr u wavr 4: The choice of 4 as a threshold value takes into account distorted images. In distorted images, pixels having low directional membership value on one direction (value 1) can be detected as part of a defect, if they have a high directional membership value 3 on the other direction. Pixels having a medium directional membership value 2 on both directions can also be detected as defect points. Pixels having low directional membership value on both directions are considered as noise, isolated noise, or background ¯uctuations. The vertical direction ®ltering described above detects round-like and transverse defects in low contrast and noisy images. For detection of longitudinal defects (including round-like defects) we use horizontal ®ltering. Reinforcement and irregularities on the weld surface cause the existence of dark weld boundaries in the horizontal direction, which makes it more dif®cult to detect imperfections. The variation of brightness in the horizontal direction is higher than in the vertical direction and therefore horizontal ®ltering requires more careful consideration. 3.2. Horizontal ®ltering For the horizontal processing we employ the horizontal ®lter shown in Fig. 12. By A En we denote the average value of the region En. By A Dm we denote the average value of the region Dm which is located m pixels away from the center in the test direction and centered at (i, j). In horizontal ®ltering, only the left and right directions of the gradient at the pixel (i0, j0) are considered. For (i, j) we gradient direction
t esting direction
En
Dm
(i 0, j 0)
n
m (i, j)
Fig. 12. Horizontal ®lter.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269 Table 3 Fuzzy rules for horizontal ®ltering (a) d M
V
C
VS S M1 M2 M3 M4 M5 L1 L2 L3
(b) d S
V
VS
S
M1
M2
M3
M4
M5
L1
L2
L3
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 2 1 0 0 0 0 0 0 0
2 2 2 1 0 0 0 0 0 0
2 2 2 2 1 0 0 0 0 0
2 2 2 2 2 1 0 0 0 0
2 2 2 2 2 2 1 0 0 0
3 3 3 3 3 3 2 1 0 0
3 3 3 3 3 3 3 2 1 0
3 3 3 3 3 3 3 3 2 1
C
S ML
S
M
0 0
1 0
de®ne spatial contrast C(i, j) in the same way as in the case of vertical processing, C i; j maxn A En 2 A Dm : We denote V1(i, j) as the spatial variance of the region Dm (the standard variance of the region Dm) and V2(i, j) as the spatial variance of the region En (the standard variance of the region En). The most dif®cult problem in the horizontal ®ltering is the detection of defects near the dark weld boundaries. A longitudinal defect located near dark weld boundaries has especially low contrast on both sides. The contrast of such defect is lower than contrast of transverse defects. To deal with such defects it was decided to increase the number of parameters. Because defects located near dark weld boundaries have low spatial variance in defect area as well in surrounding area, it was decided to employ the spatial variance of the region Dm as well the spatial variance of the region En. The evaluation of the defect membership value for a pixel in the horizontal direction is based on the following local characteristics: C, V1, V2, and the distance d n 1 m: To reduce the number of fuzzy rules we use a variable V max V1 ; V2 instead of V1, V2 and only two cases S and M are employed for the distance variable d. To evaluate pixels of such defects with a high score (2 or 3) and noise pixels with a low score, it was decided to expand and make a ®ner partition of M and L areas of the variables C and V. Table 3 shows two fuzzy rule matrices for
P1
P2
P1
P2
P1
horizontal processing. The directional membership values (right and left) are calculated in the same way as in the case of vertical ®ltering. One more problem in the horizontal ®ltering is to avoid detection of dark weld boundaries (see Fig. 8). Pixels belonging to dark weld boundaries have a high local contrast and can be detected as defect pixels in horizontal processing. To avoid this kind of spurious non-defect area detection we divide an input image into two parts P1, P2 such that P1 and P2 contain weld boundaries. Then we move the horizontal ®lter through P1, and evaluate pixels only in the case when the test direction is right i.e. we calculate only right directional membership values. Next, we move the horizontal ®lter through P2, and evaluate pixels only in the case when the test direction is left i.e. we calculate only left directional membership values. Finally, we consider only pixels in P2 and P1, which were already evaluated in the previous steps. For P2, we evaluate pixels in the case when the test direction is right and for P1 we evaluate pixels in the case when the test direction is left. The ¯ow of this process is shown in Fig. 13. As in the case of vertical ®ltering, the overall defect membership value of a pixel is determined as a sum of its two directional membership values, and pixels having a membership value higher than 4 are considered as belonging to a defect.
4. Experimental results We applied the proposed method to more than 100 X-ray images taken from steel butt weld parts. Only 16 images with small low contrast defects were used to train a neurofuzzy system. The training set of real data was de®ned from training images and used for tuning the parameters of the membership functions. The training set was formed from 50 training patterns. In the case of vertical ®ltering the contrast value of the defect pixels of training images varied from 5 to 12. Fig. 14 shows the tuned membership functions, which VS
S1 S2 S3
4.8 5.3 5.8 6.3
VS
M1 M2
M3
L
8.3
11.3
12.8
9.8
S1 S2 S3
M1
M2
M3
2.4 2.8
4.1
4.9
5.6
C
L
P2
1.8
Fig. 13. Horizontal processing.
267
3.4
6.4
V
Fig. 14. Membership functions in vertical processing.
268
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
VS S M1 M2
M3 M4
M5
L1
L2
S
L3
2.7 3.1 3.3 3.7
VS
1.0
4.3
S
1.4
4.7
5.3
6.7
M1
M2 M3
M4
1.7
2.2 2.4
3.0
7.6
8.6
M5
3.8
C
9.4
L1 L2 L3
4.2 4.4 4.6
V
Fig. 15. Membership functions in horizontal processing d M:
were used for evaluating the defect membership value of pixels in vertical ®ltering. In the case of horizontal ®ltering the contrast value of the defect pixels of training images varied from 3 to 10, and crisp partition of the variable d was chosen, 5 # S # 10; 10 , M # 30: Fig. 15 shows
S
ML
3.7
C
ML
2.0
3.0
Fig. 16. Membership functions in horizontal processing d S:
tuned membership functions which were used for evaluating the defect membership value of pixels in horizontal ®ltering when d M: Fuzzy sets of C and V in the case when d S were chosen heuristically as illustrated in Fig. 16. The sizes of the regions E and D in the vertical direction were selected as 20 £ 2 and 4 £ 7, respectively. The sizes of the regions E and D in the horizontal direction were selected as 15 £ 2 and 7 £ 3, respectively. The sizes of regions could be different, since the neuro-fuzzy system has a learning ability and so can be retrained. After fuzzy ®ltering and detecting defect pixels, methods of tracking and region growing [1] were applied to extract position and shape of defects. In practice, the image analysis system must be able to
Crack
Cracks
Crack
a) Enhanced test image (transverse cracks)
b) Result of processing of a)
V
c) Enhanced test image(longitudinal cracks)
d) Result of processing of c)
Fig. 17. Sample images and results of processing.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
recognize defect indications in the weld radiograph independent of location, shape, and defect type. In order to test the reliability of the proposed algorithm, a comparison between visual and automated evaluation was done. The reliability test was based on 117 weld radiographs with typical defects (e.g. longitudinal and transverse cracks, lack of fusion, incomplete penetration, blow hole and slag inclusions). Visual evaluation of all weld radiographs yielded altogether 181 defects. The proposed algorithm successfully detects 177 defects. Example results of the proposed method to two test images are shown in Fig. 17. Only a few small defects (one was lack of fusion and the others were slag inclusions) located near dark weld boundaries were not detected by the proposed algorithm. On 35 images misinterpretations occurred: in these cases, small noise regions were falsely recognized as defects. Noise misinterpretations are especially dif®cult to resolve since there is no strict de®nition of what a small defect or noise is. For a defect detection system it is very important to have minimal loss on defect regions even at the cost of increasing the number of non-defect areas. Detection of a few spurious non-defect areas will not cause serious problems for defect detection supporting systems, which help inspectors quickly interpret the X-ray ®lms. The number of errors of noise interpretation can be reduced by increasing the number of training samples and the number of partitions of fuzzy sets. Also, with the help of a further feature extraction it should be possible to separate and suppress noise misinterpretations. 5. Conclusions In this research, a fuzzy algorithm has been developed for the purpose of small object detection in low contrast X-ray images. By the proposed algorithm, images are ®ltered by applying fuzzy reasoning based on local image characteristics such as spatial contrast and spatial variance. The proposed method avoids many drawbacks of the conventional approaches. It makes thresholding process more smooth and ¯exible and detects as well low contrast distorted objects as non-distorted high contrast objects. The proposed algorithm was applied to detect internal weld defects from radiographic ®lms of steel butt weld parts. Results show success in detecting small low-contrast defects at a similar level to human vision. Fuzzy reasoning permits reliable and ¯exible defect recognition in weld radiographs. Future work will focus on employing automatic fuzzy partitioning techniques and development of the proposed method for other practical uses.
269
References [1] A. Rosenfeld, A. Kak, Digital Picture Processing, Academic Press, New York, 1976. [2] T. Caelli, Advances in Medical Imaging, ICPR '98, 1998, pp. 505±508. [3] W. Danm, P. Rose, H. Heidt, J. Boriljies, Automatic recognition of weld defects in X-ray inspection, British Journal of NDT 29 (2) (1987) 79±82. [4] P.Y. Glorennec, A Neuro-fuzzy inference system designed for implementation on a neural chip, Proceedings of Iizuka'92, Iizuka, Japan, 1992. [5] P.Y. Glorennec, Learning algorithms for neuro-fuzzy networks, Fuzzy Control Systems, CRC Press, Boca Raton, FL, 1994 (4±17). [6] S. Horikawa, T. Furuhashi, Y. Uchikawa, On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm, IEEE Transactions on Neural Network 3 (5) (1992) 801±806. [7] H. Ichihashi, T. Watanabe, A learning method based on simpli®ed fuzzy reasoning, Journal of Japan Society for Fuzzy Theory and Systems 2 (3) (1990) 429±437 (in Japanese). [8] A. Ishii, V. Lashkia, Y. Ochi, An image processing algorithm for small defect detection, Japanese Journal of SNDI 42 (4) (1993) 199±205 (in Japanese). [9] A. Ishii, V. Lashkia, Y. Ochi, M. Akutsu, Recognition of internal weld defects by defect model, Transaction of the Japan Society of Mechanical Engineers 60 (578) (1994) 2440±2445 (in Japanese). [10] A. Jain, M. Dubuisson, Segmentation of X-ray and C-scan images of ®ber reinforced composite materials, Pattern Recognition 25 (3) (1992) 257±270. [11] Y. Kato, T. Okumura, K. Itoga, T. Harada, K. Sugimoto, K. Michiba, S. Iuchi, Development of the automatic system for radiographic ®lm interpretation, Japanese Journal of SNDI 41 (4) (1992) 196±206. [12] J. Keller, R. Yager, Neural network implementation of fuzzy logic, Fuzzy Sets and Systems 45 (1) (1992) 1±12. [13] T. Law, H. Itoh, H. Seki, Image ®ltering, edge detection, and edge tracing using fuzzy reasoning, IEEE Transactions on PAMI 18 (5) (1996) 481±491. [14] D. Liang, W. Zhen, G. Zhang, L. Qi, Y. Tong, J. Hu, Automatical identi®cation of the defect level of welding seam based on X-ray image, Proceedings of International Symposium on Nondestructive Testing and Stress±Strain Measurement, Tokyo, 1992, pp. 267±274. [15] R. Murakami, Detection and classi®cation of welding defects in the xray ®lms by using image processing and neural network, IAPR QCAV '98, 1998, pp. 480±484. [16] H. Nomura, I. Hayashi, N. Wakami, A self-tuning method of fuzzy reasoning by delta rule and its application to moving obstacle avoidance, Journal of Japan Society for Fuzzy Theory and Systems 4 (2) (1992) 379±388 (in Japanese). [17] Y. Shirai, Automatic inspection of X-ray photograph of welding, Pattern Recognition 1 (1967) 257±261. [18] C. Tao, W. Thompson, J. Taur, A fuzzy If±Then approach to edge detection, Proceesings of the Second IEEE International Conference on Fuzzy Systems, 1993, pp. 1356±1360. [19] L.A. Zadeh, The concept of a linguistic variable and its applications to approximate reasoning Ð I, II, III, Information Science 8 (1975) 9.
www.elsevier.com/locate/imavis
Defect detection in X-ray images using fuzzy reasoning V. Lashkia* Department of Informatics and Computer Engineering, Okayama University of Science, 1-1 Ridai-cho, Okayama, 700-0005, Japan Received 19 July 1999; revised 1 August 2000; accepted 4 September 2000
Abstract Since X-ray images contain sensory noise, and objects in X-ray images are often distorted by various effects such as uneven lighting, classical image processing techniques and methods based on ordinary crisp set theory are poor at detecting small low contrast objects. In this paper, we propose a more effective method based on fuzzy theory. With the proposed algorithm, images are ®ltered by applying fuzzy reasoning using local image characteristics. The proposed algorithm was applied to detect internal weld defects from radiographic ®lms, which are taken from steel butt weld parts. Results show success in detecting defects at a similar level to human vision. A comparison between a visual and an automatic evaluation demonstrates the ef®ciency of this method. q 2001 Elsevier Science B.V. All rights reserved. Keywords: Non-destructive testing; Visual inspection; Defect detection; Fuzzy reasoning; X-ray radiography; Weld defect
1. Introduction The problem of determining what is and what is not an object in X-ray images is compounded by the fact that objects are often distorted. The objects are often seen to contain a number of regions having fairly distinct gray levels superimposed on the background level. Furthermore, X-ray images frequently contain data with object-like characteristics, such as edges with a high gradient and noise. Images are also corrupted by non-uniform illumination. In the image, most objects can be seen as a zone whose brightness is different from the surrounding area. A human can easily ®nd such a zone or boundary even when the image is corrupted by non-uniform illumination, contains noise and has low contrast. However, most of the current image processing algorithms encounter dif®culties in detecting such zones. A method that can successfully detect objects in X-ray images has many applications in medicine (detection of distortions in mammograms, etc. [2]) and in non-destructive testing of industrial products (detection of defects in welding seams [3,8±11,14,15,17] and composite materials [10]). In this paper, we propose a new method for small low contrast object detection based on fuzzy theory and apply it to the detection of defects in X-ray images of steel butt weld parts. Digital image processing in industrial radiography is used primarily for preprocessing and restoration of X-ray images * Tel.: 181-86-256-9592; fax: 181-86-255-3611. E-mail address: [email protected] (V. Lashkia).
[3,14]. Detection and classi®cation of weld defects is still done by a human observer. The non-destructive testing of welding seams involves a series of processes from sticking a ®lm-pack piece by piece along the run of the welding seam, exposing the pieces of ®lm with X-rays, to developing them to be identi®ed by humans. Because many pieces of X-ray ®lm need to be interpreted in a short time and some of the defects like transverse cracks are dif®cult to see, an automatic test approach is highly desirable. Current processing algorithms for defect detection in Xray images suffer from a number of problems. Algorithms that use smoothing operators [11,14,17] have dif®culties in detecting small objects such as cracks, which appear as a subtle change of shade in the image. When these operators are used the small low contrast objects become blurred or can completely disappear. Algorithms based on differential image calculation (between the original image and background image without defects) [3,11] are also unable to deal with small low contrast defects, as the processing is strongly affected by the size and contrast of a defect. Edge detecting techniques [10,14] are found to be effective only when there is signi®cant contrast. Attempts to detect edges of small low contrast defects makes the method sensitive to noise. Conventional image segmentation methods such as adaptive thresholding [10,15] or iterated conditional modes [10] are unable to properly segment non-uniformly illuminated images. To overcome non-uniformly illuminated images. To overcome non-uniform illumination problems a new adaptive threshold method based on Yanowitz and Bruckstein's algorithm with a Canny edge detector was
0262-8856/01/$ - see front matter q 2001 Elsevier Science B.V. All rights reserved. PII: S 0262-885 6(00)00075-5
262
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
Fig. 1. Incomplete penetration (image is magni®ed).
proposed in Ref. [10]. This method uses edge points to interpolate the threshold surface, and works well for relatively large defects but does not differentiate between small defects and noise. In Refs. [8,9], an adaptive model-based defect detection method RP was introduced. RP detects objects by ®tting to them a ªconcentric netº model, which is built from radiated rays and concentric curves. By evaluating local statistical parameters, such as spatial contrast and variance, and employing commonsense reasoning rules, which may resemble the heuristic rules employed by humans, RP adjusts its frames. Also, unlike many other recognition schemes it does not rely on some form of pre-normalization of input images, and can handle distorted low contrast images as well as non-distorted high contrast images. Although this method shows good experimental results, it requires many parameters and threshold values that must be set empirically. In an attempt to improve the performance of the RP method we introduce a new approach which implements fuzzy reasoning using a neuro-fuzzy system, and which makes the thresholding process smoother and more ¯exible. Since small low contrast defects include fatal defects (cracks), the performance of the defect detection method needs to be at the same level as that of ®eld inspectors. To perceive objects in imperfect images, it appears that humans are applying heuristic algorithms to understand such images [13]. We believe that it is possible to characterize an equivalent process systematically. We propose that fuzzy reasoning is a suitable framework for expressing heuristic processes applied to imperfect data. Fuzzy reasoning, as outlined by Zadeh [19], with the power to model and respond usefully to approximate situations, is ideally suited to this problem. The proposed method acknowledges the existence of distorted low contrast defects as well as non-distorted high contrast defects, and elaborates a linguistic framework for characterizing these imperfections. These characteristics are
approximated from local statistical parameters (such as spatial contrast and spatial variance), and fuzzy rules based on heuristics are introduced to evaluate these parameters. Analogous approaches have been introduced in Refs. [13,18], where fuzzy reasoning based on local statistical parameters was successfully applied to edge detection, achieving better performance than conventional crisp methods. Here, we report the ®rst attempt to detect low contrast defects in X-ray ®lms with a fuzzy reasoning approach. The major advantages of the proposed method are as follows: (1) it is based on commonsense fuzzy reasoning rules, which offer an intrinsic understanding of the classi®cation logic; (2) the parameters and threshold values are not selected empirically as in many conventional methods, but are obtained by the training process; (3) the method is simple (only values of spatial contrast and spatial variance are used to determine the existence of a defect); (4) it is ¯exible (the method works for distorted low contrast images as well as non-distorted high contrast images); and (5) it is stable (the proposed method was tested on more than 100 images, and results show success in detecting defects at a similar level to human vision). This paper is organized as follows. In Section 2, we describe the principles behind the proposed method. In Section 3, we apply the proposed method to the problem of defect detection in X-ray images of steel butt weld parts. In Section 4, we discuss experimental results. 2. Proposed fuzzy reasoning In X-ray images, small objects can often be seen as a zone with low contrast. Because noise in X-ray images has a spotty contrast, small objects cannot be detected by methods based only on contrast characteristics. On the other hand, although noise can be seen as a zone with high spatial variance, methods based on this feature are ineffective because objects exhibit the same features due to distortion and uneven lighting. However, the combination of contrast and variance could be used to distinguish objects from noise. This is because an object is different to noise in that it has low spatial variance when the contrast is low (and therefore appears as a zone whose brightness is different from the surrounding area). In Fig. 1, we show an example of a defect in a steel butt weld part. The defect has low contrast and low spatial variance, as shown in Fig. 2. If the contrast increases, an object can have intensity variation (Figs. 3 and 4), and the higher the contrast the easier the object is to detect. These ideas are expressed as a 3D surface in Fig. 5. For a ®xed contrast value, the lower the variance the more con®dent we can be that the observed area is an object. For a ®xed variance value, the higher the contrast the more con®dent we can be that the observed area is an object. The proposed method of fuzzy reasoning is based on Fig. 5 and fuzzy rules of the proposed defect detection system are formed according to it.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
Fig. 2. 3D representation of the image from Fig. 1.
One more parameter which is useful for object detection in X-ray images is the distance between two regions characterizing the contrast parameter. Our investigations of noise areas in X-ray images show that contrast regions separated by short distances indicate a strong possibility of noise. On the other hand, for medium distances the contrast of noise area becomes low, and for long distances the distinction between the object and noise is confusing. In Fig. 6 we show an area of an X-ray image of a steel butt weld part containing no defects. As seen from Fig. 6, there is an intensity variation in the small region that surrounds particular isolated noise, although the difference between the intensity values of the isolated noise and an area that
263
Fig. 4. 3D representation of the image from Fig. 3.
is relatively far from it is, in general small, because almost all areas contain some noise pixels. This suggests that it is preferable to have higher con®dence of an object for medium distances than for short distances. The proposed method evaluates the object fuzzy membership value for each pixel, based on the local image characteristics described above. Pixels having high membership are detected as regions belonging to defects and then tracking and region growing methods are applied to detect the exact position and shape of the defects. To calculate a fuzzy variable we employ the simpli®ed fuzzy reasoning method proposed in Ref. [7]. The simpli®ed fuzzy reasoning method induces a neuro-fuzzy system, which was investigated in great detail in Refs. [4,6,12,16]. When the input of neuro-fuzzy system is de®ned as x1 ; ¼; xm and the output is y, the ith rule of the simpli®ed Membership for object
0 Contrast 0 Fig. 3. Slag inclusion (image is magni®ed).
Variance
Fig. 5. Control surface of the proposed fuzzy reasoning.
0
264
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
Fig. 7. A de®nition of a spatial contrast and variance.
average or a medium value of Fu : The value X Fu i; j F i; j
1
u
Fig. 6. Noise area (image is magni®ed).
fuzzy reasoning is given by the following expression: if x1 is Ai1 ¼ and xm is Aim then y is wi; where i i 1; ¼; n is the rule number, Ai1 ; ¼; Aim are membership functions, and wi is the real value of the consequent part. The output y of the neuro-fuzzy system is calculated as outlined below n X
y
wi mi
i1 n X i1
;
mi Ai1 x1 ¼Aim xm :
mi
Several methods for tuning parameters of membership functions of a neuro-fuzzy system were proposed. In this research we employ the method of steepest descent described in Refs. [5,7]. The proposed fuzzy reasoning approach can be implemented in various ways. Below, we present one of the possible implementations. The spatial contrast Cu i; j can be de®ned as a difference in the average values of pixels of regions E and D, see Fig. 7. The spatial variance Vu i; j can be de®ned as the standard variance of the region D. The size of regions D and E depends on the minimal size of the defect which we want to detect. Denote by Fu i; j the membership for an object calculated based on Cu ; Vu ; and n using a neuro-fuzzy system. We call Fu the object u -directional membership value. We determine if the i; j pixel is a part of the object or background, based on the values Fu i; j: A classi®er based approach (for example neural network) can be employed for this purpose. Another simple way to handle the values Fu is to employ a thresholding method. The threshold value can be chosen in various ways depending on the objects we are looking for. One possible way, which takes into account distorted and low contrast objects, is as follows. Suppose that wavr is an
we call the overall object membership value of the pixel i; j: We choose a threshold value X wavr ; 2 tr u
and may assume i; j is an object pixel if F i; j $ tr: We describe a more detailed implementation of the proposed method in Section 3. The important part of the proposed method is the training procedure. We train a neuro-fuzzy system using a training set, which is formed from small low contrast objects and noise. In this paper, we employ triangular and trapezoidal membership functions and use the following fuzzy set values: very small (VS), small (S), medium (M), and large (L). For a variable x we de®ne the fuzzy set VS as an area to the left of the minimal value of x from the object training samples. We also de®ne the fuzzy set L as an area to the right of the maximal value of x from the object training Weld boundaries
Pencil mark
Crack
512pixels
Fig. 8. A sample X-ray image of a welding seam.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
265
Grey value
Radiographic Film
1
crack
Pixels 512
ImageSampling and Gray-Level Quantization
Fig. 9. Plot of gray values along the defect.
samples. The area between VS and L is represented by fuzzy sets S and M.
FuzzyReasoning
Fuzzy Reasoning
(Vertical)
(Horizontal)
+
3. Application to defect detection DefectIsolation
The proposed method was applied to the problem of detecting internal weld defects from radiographic ®lms taken from steel butt weld zones. All radiographs are digitized by a TV-camera and stored in the computer. The resolution is 512 £ 480 pixels with 256 gray levels. Fig. 8 shows a digitized sample image of a transverse crack. The bright region in Fig. 8 is the weld zone. In Fig. 9, a plot of the gray values along the line of the crack is shown. The variation of the gray value of noise and the defect area is almost the same. Some of defects (such as transverse cracks) are dif®cult to ®nd even by human vision. Table 1 shows the names and visual features of the main weld defects. Various shapes of weld defects were found but we can roughly classify them into three classes. These are round-like defects (blow hole, slag inclusion, incomplete penetration), longitudinal defects (lack of fusion, incomplete penetration, crack) and transverse defects (cracks). Taking into account the possible shapes of defects it is reasonable to ®lter images by applying fuzzy reasoning in only the horizontal and vertical directions. In horizontal ®ltering we concentrate on the detection of longitudinal defects and in vertical ®ltering we concentrate on the detection of transverse defects. Round-like defects would be detected in both directions. Fig. 10 shows a ¯ow chart of the automatic detection of weld defects. The algorithm consists of the following parts: ² In the ®rst step, a region of interest on the weld radioTable 1 Names and visual features of the main weld defects Defect
Visual representation
Blow hole Slag inclusion Lack of fusion
Roundish zone having high contrast Large spread zone Long and narrow zone located between the center and boundary of weld zone Long and narrow zone located in the center of weld zone Thread-like zone with low contrast
Incomplete penetration Crack
Result
Fig. 10. Flow chart for the automatic detection of weld defects.
graph is digitized by a TV-camera and stored in the computer. ² The main part of the algorithm is the defect detection procedure. By applying fuzzy reasoning in vertical and horizontal directions the positions of defect kernels are detected. ² Tracking and region growing methods are applied to detect the exact position and shape of the defects. The result of the image analysis procedure is a binary image, where the background is white and defect indications are black.
3.1. Vertical ®ltering For the vertical processing, we use spatial contrast C and spatial variance V to evaluate the defect fuzzy membership value. The spatial contrast is determined by the characteristics of two regions one on either side of the center i0 ; j0 of the vertical ®lter shown in Fig. 11. To ignore noise effects Gradient direction En n (i 0, j 0) D (i, j) Testing direction
Fig. 11. Vertical ®lter.
266
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
we consider the average value of pixels in each region. By A En we denote the average value of the region En which is located n pixels away from the center in the direction of the gradient. By A D we denote the average value of the region D which is located in the direction opposite to the gradient (we call this the testing direction, because defect pixels are assumed to be in this direction) and centered at (i, j). In vertical ®ltering only up and down directions of the gradient at the pixel (i0, i0) are considered. For the (i, j) pixel we de®ne the spatial contrast as C i; j maxn A En 2 A D and the spatial variance V i; j as the standard variance of the region D. Examining noise pixels in sample images, we found that the spatial contrast values of noise pixels are concentrated in S and M areas of the defect pixel contrast values. Therefore, to achieve a high defect detection performance we choose narrower fuzzy set values and ®ner partitions for the S and M fuzzy values of variables C and V. The S and M fuzzy sets were divided into S1, S2, S3 and M1, M2, M3 fuzzy sets, respectively. According to Fig. 5 we set the inference rules of the simpli®ed fuzzy reasoning, as shown in Table 2, and use it to calculate the defect membership value. The values 1, 2, and 3 in Table 2 express the low, medium, and high con®dence accordingly. When spatial variance is low and spatial contrast is large, medium and small, a pixel is considered as a part of a defect with the membership value 3, 2 and 1, respectively as shown in Table 2. The defect membership value decreases when the spatial variance increases. The choice of defect membership values is made by taking into account distorted images and will be explained below. For a pixel (i, j) the proposed method evaluates the defect membership value based on rules shown in Table 2. Denote f1 i; j as the defect membership value calculated for (i, j) when the test direction is down. Denote f2 i; j as the defect membership value calculated for (i, j) when the test direction is up. When the ®lter across an image is moved, each pixel defect membership value can be calculated several times. For each pixel we calculate the down directional membership value F1 (i.e. Fu , where u p=2) as the maximum of membership values calculated for a pixel when the test direction was down, F1 i; j max f1 i; j: For each pixel we calculate the up directional membership value F2 (i.e. Fu , where u 2p=2) as the maximum of membership values calculated over a pixel when the test direction was up, F1 i; j max f2 i; j: According to Eq. (1), the overall defect membership value F of a pixel (i, j) is determined as a sum of its directional membership values F i; j F1 i; j 1 F2 i; j: The reason for using the maximum function in calculating F1 and F2 is to give almost the same defect membership value for pixels of small or low contrast defects as pixels of large or high contrast defects. The use of summation instead of maximum value makes thresholding (deciding pixels are part of a defect) complicated. Let wavr 2; which corresponds to the medium value of Fu . According to Eq. (2), a pixel is considered a part of a
Table 2 Fuzzy rules for vertical ®ltering C
V
VS S1 S2 S3 M1 M2 M3 L
VS
S1
S2
S3
M1
M2
M3
L
0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0
1 1 0 0 0 0 0 0
1 1 1 0 0 0 0 0
2 2 2 2 1 0 0 0
2 2 2 2 2 1 0 0
2 2 2 2 2 2 1 0
3 3 3 3 3 3 2 1
defect if and only if the overall membership P value of the pixel is greater than or equal to tr u wavr 4: The choice of 4 as a threshold value takes into account distorted images. In distorted images, pixels having low directional membership value on one direction (value 1) can be detected as part of a defect, if they have a high directional membership value 3 on the other direction. Pixels having a medium directional membership value 2 on both directions can also be detected as defect points. Pixels having low directional membership value on both directions are considered as noise, isolated noise, or background ¯uctuations. The vertical direction ®ltering described above detects round-like and transverse defects in low contrast and noisy images. For detection of longitudinal defects (including round-like defects) we use horizontal ®ltering. Reinforcement and irregularities on the weld surface cause the existence of dark weld boundaries in the horizontal direction, which makes it more dif®cult to detect imperfections. The variation of brightness in the horizontal direction is higher than in the vertical direction and therefore horizontal ®ltering requires more careful consideration. 3.2. Horizontal ®ltering For the horizontal processing we employ the horizontal ®lter shown in Fig. 12. By A En we denote the average value of the region En. By A Dm we denote the average value of the region Dm which is located m pixels away from the center in the test direction and centered at (i, j). In horizontal ®ltering, only the left and right directions of the gradient at the pixel (i0, j0) are considered. For (i, j) we gradient direction
t esting direction
En
Dm
(i 0, j 0)
n
m (i, j)
Fig. 12. Horizontal ®lter.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269 Table 3 Fuzzy rules for horizontal ®ltering (a) d M
V
C
VS S M1 M2 M3 M4 M5 L1 L2 L3
(b) d S
V
VS
S
M1
M2
M3
M4
M5
L1
L2
L3
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 2 1 0 0 0 0 0 0 0
2 2 2 1 0 0 0 0 0 0
2 2 2 2 1 0 0 0 0 0
2 2 2 2 2 1 0 0 0 0
2 2 2 2 2 2 1 0 0 0
3 3 3 3 3 3 2 1 0 0
3 3 3 3 3 3 3 2 1 0
3 3 3 3 3 3 3 3 2 1
C
S ML
S
M
0 0
1 0
de®ne spatial contrast C(i, j) in the same way as in the case of vertical processing, C i; j maxn A En 2 A Dm : We denote V1(i, j) as the spatial variance of the region Dm (the standard variance of the region Dm) and V2(i, j) as the spatial variance of the region En (the standard variance of the region En). The most dif®cult problem in the horizontal ®ltering is the detection of defects near the dark weld boundaries. A longitudinal defect located near dark weld boundaries has especially low contrast on both sides. The contrast of such defect is lower than contrast of transverse defects. To deal with such defects it was decided to increase the number of parameters. Because defects located near dark weld boundaries have low spatial variance in defect area as well in surrounding area, it was decided to employ the spatial variance of the region Dm as well the spatial variance of the region En. The evaluation of the defect membership value for a pixel in the horizontal direction is based on the following local characteristics: C, V1, V2, and the distance d n 1 m: To reduce the number of fuzzy rules we use a variable V max V1 ; V2 instead of V1, V2 and only two cases S and M are employed for the distance variable d. To evaluate pixels of such defects with a high score (2 or 3) and noise pixels with a low score, it was decided to expand and make a ®ner partition of M and L areas of the variables C and V. Table 3 shows two fuzzy rule matrices for
P1
P2
P1
P2
P1
horizontal processing. The directional membership values (right and left) are calculated in the same way as in the case of vertical ®ltering. One more problem in the horizontal ®ltering is to avoid detection of dark weld boundaries (see Fig. 8). Pixels belonging to dark weld boundaries have a high local contrast and can be detected as defect pixels in horizontal processing. To avoid this kind of spurious non-defect area detection we divide an input image into two parts P1, P2 such that P1 and P2 contain weld boundaries. Then we move the horizontal ®lter through P1, and evaluate pixels only in the case when the test direction is right i.e. we calculate only right directional membership values. Next, we move the horizontal ®lter through P2, and evaluate pixels only in the case when the test direction is left i.e. we calculate only left directional membership values. Finally, we consider only pixels in P2 and P1, which were already evaluated in the previous steps. For P2, we evaluate pixels in the case when the test direction is right and for P1 we evaluate pixels in the case when the test direction is left. The ¯ow of this process is shown in Fig. 13. As in the case of vertical ®ltering, the overall defect membership value of a pixel is determined as a sum of its two directional membership values, and pixels having a membership value higher than 4 are considered as belonging to a defect.
4. Experimental results We applied the proposed method to more than 100 X-ray images taken from steel butt weld parts. Only 16 images with small low contrast defects were used to train a neurofuzzy system. The training set of real data was de®ned from training images and used for tuning the parameters of the membership functions. The training set was formed from 50 training patterns. In the case of vertical ®ltering the contrast value of the defect pixels of training images varied from 5 to 12. Fig. 14 shows the tuned membership functions, which VS
S1 S2 S3
4.8 5.3 5.8 6.3
VS
M1 M2
M3
L
8.3
11.3
12.8
9.8
S1 S2 S3
M1
M2
M3
2.4 2.8
4.1
4.9
5.6
C
L
P2
1.8
Fig. 13. Horizontal processing.
267
3.4
6.4
V
Fig. 14. Membership functions in vertical processing.
268
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
VS S M1 M2
M3 M4
M5
L1
L2
S
L3
2.7 3.1 3.3 3.7
VS
1.0
4.3
S
1.4
4.7
5.3
6.7
M1
M2 M3
M4
1.7
2.2 2.4
3.0
7.6
8.6
M5
3.8
C
9.4
L1 L2 L3
4.2 4.4 4.6
V
Fig. 15. Membership functions in horizontal processing d M:
were used for evaluating the defect membership value of pixels in vertical ®ltering. In the case of horizontal ®ltering the contrast value of the defect pixels of training images varied from 3 to 10, and crisp partition of the variable d was chosen, 5 # S # 10; 10 , M # 30: Fig. 15 shows
S
ML
3.7
C
ML
2.0
3.0
Fig. 16. Membership functions in horizontal processing d S:
tuned membership functions which were used for evaluating the defect membership value of pixels in horizontal ®ltering when d M: Fuzzy sets of C and V in the case when d S were chosen heuristically as illustrated in Fig. 16. The sizes of the regions E and D in the vertical direction were selected as 20 £ 2 and 4 £ 7, respectively. The sizes of the regions E and D in the horizontal direction were selected as 15 £ 2 and 7 £ 3, respectively. The sizes of regions could be different, since the neuro-fuzzy system has a learning ability and so can be retrained. After fuzzy ®ltering and detecting defect pixels, methods of tracking and region growing [1] were applied to extract position and shape of defects. In practice, the image analysis system must be able to
Crack
Cracks
Crack
a) Enhanced test image (transverse cracks)
b) Result of processing of a)
V
c) Enhanced test image(longitudinal cracks)
d) Result of processing of c)
Fig. 17. Sample images and results of processing.
V. Lashkia / Image and Vision Computing 19 (2001) 261±269
recognize defect indications in the weld radiograph independent of location, shape, and defect type. In order to test the reliability of the proposed algorithm, a comparison between visual and automated evaluation was done. The reliability test was based on 117 weld radiographs with typical defects (e.g. longitudinal and transverse cracks, lack of fusion, incomplete penetration, blow hole and slag inclusions). Visual evaluation of all weld radiographs yielded altogether 181 defects. The proposed algorithm successfully detects 177 defects. Example results of the proposed method to two test images are shown in Fig. 17. Only a few small defects (one was lack of fusion and the others were slag inclusions) located near dark weld boundaries were not detected by the proposed algorithm. On 35 images misinterpretations occurred: in these cases, small noise regions were falsely recognized as defects. Noise misinterpretations are especially dif®cult to resolve since there is no strict de®nition of what a small defect or noise is. For a defect detection system it is very important to have minimal loss on defect regions even at the cost of increasing the number of non-defect areas. Detection of a few spurious non-defect areas will not cause serious problems for defect detection supporting systems, which help inspectors quickly interpret the X-ray ®lms. The number of errors of noise interpretation can be reduced by increasing the number of training samples and the number of partitions of fuzzy sets. Also, with the help of a further feature extraction it should be possible to separate and suppress noise misinterpretations. 5. Conclusions In this research, a fuzzy algorithm has been developed for the purpose of small object detection in low contrast X-ray images. By the proposed algorithm, images are ®ltered by applying fuzzy reasoning based on local image characteristics such as spatial contrast and spatial variance. The proposed method avoids many drawbacks of the conventional approaches. It makes thresholding process more smooth and ¯exible and detects as well low contrast distorted objects as non-distorted high contrast objects. The proposed algorithm was applied to detect internal weld defects from radiographic ®lms of steel butt weld parts. Results show success in detecting small low-contrast defects at a similar level to human vision. Fuzzy reasoning permits reliable and ¯exible defect recognition in weld radiographs. Future work will focus on employing automatic fuzzy partitioning techniques and development of the proposed method for other practical uses.
269
References [1] A. Rosenfeld, A. Kak, Digital Picture Processing, Academic Press, New York, 1976. [2] T. Caelli, Advances in Medical Imaging, ICPR '98, 1998, pp. 505±508. [3] W. Danm, P. Rose, H. Heidt, J. Boriljies, Automatic recognition of weld defects in X-ray inspection, British Journal of NDT 29 (2) (1987) 79±82. [4] P.Y. Glorennec, A Neuro-fuzzy inference system designed for implementation on a neural chip, Proceedings of Iizuka'92, Iizuka, Japan, 1992. [5] P.Y. Glorennec, Learning algorithms for neuro-fuzzy networks, Fuzzy Control Systems, CRC Press, Boca Raton, FL, 1994 (4±17). [6] S. Horikawa, T. Furuhashi, Y. Uchikawa, On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm, IEEE Transactions on Neural Network 3 (5) (1992) 801±806. [7] H. Ichihashi, T. Watanabe, A learning method based on simpli®ed fuzzy reasoning, Journal of Japan Society for Fuzzy Theory and Systems 2 (3) (1990) 429±437 (in Japanese). [8] A. Ishii, V. Lashkia, Y. Ochi, An image processing algorithm for small defect detection, Japanese Journal of SNDI 42 (4) (1993) 199±205 (in Japanese). [9] A. Ishii, V. Lashkia, Y. Ochi, M. Akutsu, Recognition of internal weld defects by defect model, Transaction of the Japan Society of Mechanical Engineers 60 (578) (1994) 2440±2445 (in Japanese). [10] A. Jain, M. Dubuisson, Segmentation of X-ray and C-scan images of ®ber reinforced composite materials, Pattern Recognition 25 (3) (1992) 257±270. [11] Y. Kato, T. Okumura, K. Itoga, T. Harada, K. Sugimoto, K. Michiba, S. Iuchi, Development of the automatic system for radiographic ®lm interpretation, Japanese Journal of SNDI 41 (4) (1992) 196±206. [12] J. Keller, R. Yager, Neural network implementation of fuzzy logic, Fuzzy Sets and Systems 45 (1) (1992) 1±12. [13] T. Law, H. Itoh, H. Seki, Image ®ltering, edge detection, and edge tracing using fuzzy reasoning, IEEE Transactions on PAMI 18 (5) (1996) 481±491. [14] D. Liang, W. Zhen, G. Zhang, L. Qi, Y. Tong, J. Hu, Automatical identi®cation of the defect level of welding seam based on X-ray image, Proceedings of International Symposium on Nondestructive Testing and Stress±Strain Measurement, Tokyo, 1992, pp. 267±274. [15] R. Murakami, Detection and classi®cation of welding defects in the xray ®lms by using image processing and neural network, IAPR QCAV '98, 1998, pp. 480±484. [16] H. Nomura, I. Hayashi, N. Wakami, A self-tuning method of fuzzy reasoning by delta rule and its application to moving obstacle avoidance, Journal of Japan Society for Fuzzy Theory and Systems 4 (2) (1992) 379±388 (in Japanese). [17] Y. Shirai, Automatic inspection of X-ray photograph of welding, Pattern Recognition 1 (1967) 257±261. [18] C. Tao, W. Thompson, J. Taur, A fuzzy If±Then approach to edge detection, Proceesings of the Second IEEE International Conference on Fuzzy Systems, 1993, pp. 1356±1360. [19] L.A. Zadeh, The concept of a linguistic variable and its applications to approximate reasoning Ð I, II, III, Information Science 8 (1975) 9.