- Email: [email protected]
Visual inspection of products with geometrical quality characteristics of known tolerances
Ain Shams Engineering Journal (2010) 1, 79–84
Ain Shams University
Ain Shams Engineering Journal www.elsevier.com/locate/asej www.sciencedirect.com
MECHANICAL ENGINEERING
Visual inspection of products with geometrical quality characteristics of known tolerances Mohammed H. Hassan a b
a,*
, Safaa L. Diab
b
Helwan University, Faculty of Engineering, Helwan, Cairo, Egypt Helwan University, Faculty of Science, Helwan, Cairo, Egypt
Received 24 April 2010; accepted 30 June 2010 Available online 5 November 2010
KEYWORDS Visual inspection; Image processing; Thresholding; Tolerance; Regression; ANOVA
Abstract Image analysis techniques are being increasingly used to automate industrial inspection. The manual activity of inspection could be subjective and highly dependent on the experience of human personnel. In a previous work, the authors presented two approaches utilizing image analysis to inspect products visually. The product is accepted or rejected based on its conformance to specified tolerances, where conformance is analogy measured through statistical indices like correlation and root mean square error. In this work, the authors introduce a novel visual inspection approach that can be used on line to test simultaneously multiple quality characteristics. The approach utilizes image processing tools to deal with the product image; and extract features of its geometrical characteristics. Based on tolerance bands of each characteristic, an index is experimentally developed to reflect the deviation of a quality characteristic dimension from its nominal value; and one can decide whether a characteristic complies with the pre-specified tolerance. Statistical analysis proved that there is a strong association between the developed indices and the deviations of quality characteristics from their target values. Linear regression models are proved to model these associations; and are used to give the corresponding indices’ values relative to the tolerance specifications. The developed approach is
* Corresponding author. E-mail addresses: [email protected] Hassan), [email protected] (S.L. Diab).
(M.H.
2090-4479 Ó 2010 Ain Shams University. Production and hosting by Elsevier B.V. All rights reserved. Peer review under responsibility of Ain Shams University. doi:10.1016/j.asej.2010.09.011
Production and hosting by Elsevier
80
M.H. Hassan, S.L. Diab proved to give a good performance experimentally in detecting non-conforming products; and, in specific, the defect location(s). Ó 2010 Ain Shams University. Production and hosting by Elsevier B.V. All rights reserved.
1. Introduction The rapidly increase in automating production processes should be sustained by automated inspection technologies. Such technologies can help reducing traditional quality control costs and errors; and in turn, eliminating wastes. Inspection is defined as the process of determining whether products deviate from a given set of specifications [13,26]. Manufacturing has long recognized that 100% inspection using a human inspector does not find 100% of the defective items. Since this situation was recognized, inspection has been altered from 100% inspection to sampling inspection using statistical quality control (SQC) which was later incorporated more directly into the production processes using statistical process control (SPC). While well implemented SPC and SQC programs can drastically reduce the fraction of defective products reaching the customer, they could never be perfect for all intended purposes [25]. Therefore, developing an automated visual inspection system will significantly contribute to the quality improvement of products and reduce chances for passing defected parts to next production phases. Automated visual inspection determines the properties of products using visual information and is most often automated by employing machine vision techniques. With recent advances in computer technology, modern manufacturers have turned their attention to machine-vision inspection systems. Such modern devices now inspect metal products [1,15], textile fabrics [19], pipeline [9,24]; and are used in trash separation processes [20], and robotics [6] even while products are on conveyors. In a previous work, the authors [11] introduced statistically based visual approaches that can be used on line to inspect products with multiple dimensions and known tolerances. The approaches’ output is that the inspected product is accepted or rejected based on a calculated statistical index(s). The approaches cannot recognize the location of defect(s) or which quality characteristic causes rejection. In this research, the developed approach integrates image processing approaches with statistical regression to find out a new index for each quality characteristic of the product; and accordingly, determine whether that characteristic conforms to tolerance specifications. 2. Previous work Most of the defect detection systems are focused on non-textured surfaces such as glass panel [28], sheet steel [21] and uniform web materials [2] using simple thresholding or edge detection techniques. Defects in these images can be easily detected because they commonly have distinctly measured values with respect to those of the uniform background. Automatic visual inspection techniques for textured images generally compute a set of textural features in the spatial domain or in the spectral domain, and then search for significant
local deviations in the feature values using various classifiers. In spatial-domain approaches, the commonly used features are the second-order statistics derived from spatial gray-level co-occurrence matrices [10]. They have been applied to wood inspection [5], carpet wear assessment [23], roughness measurement of machined surfaces [22], and surface defect detection [14]. Inspection of dimensional defects has become a critical task for manufacturers who strive to improve product quality and production efficiency [16,18]. Research concerning automated visual inspection grows rapidly. Leopold et al. [17] introduced new approaches in fast 3D-surface quality control. They made a good classification of these approaches and their application fields. Most of researches integrated AI approaches with image processing tools. Lin [12] developed an approach that utilizes neural network and statistical approach to inspect light-emitting diode chips. Fricout et al. [7] developed an on-line approach to inspect the smoothness of metal surfaces. The measurements were based on topographical maps obtained through interferometric microscopy. The resulting data were analyzed by an algorithm based on morphological and statistical features extraction from surface topography, factorial analysis, bootstrap over-sampling and Bayesian classification. Lin [13] presented a wavelet characteristic based approach for the automated visual inspection of ripple defects in the surface barrier layer chips of ceramic capacitors. Among the useful applications of visual inspection is the colour unevenness testing like in LCD monitors which is not easy to be made by humans as a result of human subjectivity and eye fatigue. Chiu and Lin [4] developed a hybrid approach to test blemishes in LCD panels. Their approach is based on Hotelling statistics and image analysis. Other examples includes [1,3,27,29]. The decision of all developed approaches is either to accept or reject the product without determining the source of defect(s). The approach introduced in this research utilizes image processing and statistical approaches to inspect simultaneously multiple quality characteristics. It is recognized by giving a decision for each quality characteristic such that a product defect(s) are recognized; and not just accept/reject decision. 3. The developed approach When an image is digitally transformed and acquired by a computer, it is represented by a two-dimensional matrix. The matrix elements values depend on the type of the image that can be classified as binary, colour, gray scale, etc. As by Gonzalez et al. [8], an image may be defined as a two dimensional function, f(x, y), where x and y are spatial coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of the image. The first step in this approach is to specify non-overlapping regions (called masks) over the image such that each region covers completely one of the image’s quality characteristics.
Visual inspection of products with geometrical quality characteristics of known tolerances The locations of these masks must be determined such that each quality characteristic belongs to one (and only one) mask. A good selection of masks is realized when the mask region can be easily determined its coordinates and the size of the mask is as small as possible. This is considered as an initial selection of masks. The next step is to threshold the image and transform its matrix elements into binary values (0, 1). Final coordinates of masks are specified based on the clusters of zeros and ones that determine the locations of quality characteristics in the image matrix. A final step is to calculate an index for each quality characteristic that reflects the extent of conformance of that quality characteristic to the target or nominal characteristic of the reference image (the corresponding characteristic with nominal dimensions). The index is calculated for each mask (m) using the following equation: Im ¼
nm X Si n m i¼1
m ¼ 1; 2; . . . ; M
where Si is a (0, 1) variable; which is one when the mask’s i pixel is similar to the corresponding one of the reference image and zero otherwise, nm is the number of pixels of mask m, and M represents the number of masks, or number of quality characteristics. The M indices are calculated simultaneously. Each pixel of a product’s image matrix is compared to its corresponding one of the reference image; and according to its location (the mask it belongs to), the index value of that mask is updated. The value of an index is exactly equal one (theoretically) if the quality characteristic dimensions are the same as the reference one and
ð1Þ
Figure 3
Figure 1
Circle one diameter vs. index.
The reference product image.
Figure 4
Figure 2
81
The reference image with masks.
Figure 5
Slot width vs. index.
Circle two diameter vs. index.
82 Table 1
M.H. Hassan, S.L. Diab MINITAB results of the data statistical analyses.
Regression analysis: C1 INDEX versus DC1 The regression equation is C1 INDEX = 1.00 0.0136DC1 Predictor Coef SE Coef Constant 0.999580 DC1 0.0136063 S = 0.0006363, R2 = 97.0%, R2(adj) = 97.0%
0.000063 0.0001552
T
P
15,781.33 87.69
0.000 0.000
Analysis of variance Source
DF
SS
MS
F
P
Regression Residual error Total
1 239 240
0.0031139 0.0000968 0.0032106
0.0031139 0.0000004
7689.67
0.000
Regression analysis: C2 INDEX versus DC2 The regression equation is C2 INDEX = 1.00 0.0347DC2 Predictor Coef SE Coef Constant 0.999663 DC2 0.0347327 S = 0.001384, R2 = 96.4%, R2(adj) = 96.4%
0.000096 0.0004352
T
P
10,412.70 79.82
0.000 0.000
Analysis of variance Source
DF
SS
MS
F
P
Regression Residual error Total
1 239 240
0.012208 0.000458 0.012666
0.012208 0.000002
6370.56
0.000
T
P
6388.84 51.79
0.000 0.000
Regression analysis: S1 INDEX versus DC3 The regression equation is S1 INDEX = 1.00 0.0207DC3 Predictor Coef SE Coef Constant 0.999557 DC3 0.0206534 S = 0.001608, R2 = 91.8%, R2(adj) = 91.8%
0.000156 0.0003988
Analysis of variance Source
DF
SS
MS
F
P
Regression Residual error Total
1 239 240
0.0069354 0.0006180 0.0075534
0.0069354 0.0000026
2682.21
0.000
has no shift in position. Deviation from that target downsizes the index value. Experimentally, one can determine an acceptable value lower limit for Im when the dimensions of the quality characteristic fall within the tolerance region. The approach can also be used to detect a shift in the quality characteristic position. If, for example, a hole’s centre is shifted from its correct position, the calculated index will go down even if the hole diameter is within tolerance. Calculating the acceptable Im value for different characteristics, a product can be determined whether there is a problem(s) with one or more of its quality characteristics one by one. 4. Application The developed approach is applied to the product exhibited in Fig. 1. As shown, the product has five quality characteristics which are the two small holes (4 mm diameter), the two slots (4 mm width), and the middle big hole (20 mm diameter). All dimensions of the quality characteristics assumed to have a tolerance of ±0.5 mm.
At first, the five masks are selected as shown in Fig. 2; where each mask dominates its quality characteristic. The image is then thresholded and transformed to a binary image. The final coordinates of the masks are determined based on the matrix clusters of zeros and ones. The determination of acceptable indices’ values of the quality characteristics, as well as the digital image data processing requires experimental work. In this context, the authors used 170 images with different dimensions that all fall within the tolerance specifications,70 images with some out of tolerance dimensions, and 30 images with shifted quality characteristic(s) positions. The resulted quality characteristics indices of all tested images are plotted versus the characteristics’ dimensions. Figs. 3–5 exhibit the indices of the small holes, the slots, and the large hole, respectively. As the selected masks of similar characteristics (the two small holes or the two slots) are identical, the resulted indices of these characteristics are almost the same. Accordingly, the further analyses will be concerned with three characteristics: the small hole, the big hole, and the slot. It can be concluded that one can find an index value (lower limit) for each quality characteristic that tells whether that
Visual inspection of products with geometrical quality characteristics of known tolerances characteristic conforms to specifications. This can be achieved by plotting the specification limits, as in Fig. 3, and find the corresponding index value. The figures show a high similarity of the index values around the nominal sizes. Linear regression analyses are conducted to test the relationships between the indices and the deviation from the nominal sizes of the different quality characteristics. The analyses gave a good fitting regression equations with very low fitting errors. In Table 1, the MINITAB results output for the regression analyses and the corresponding ANOVA results are exhibited. C1, C2 and S1 in Table 1 refer to the small holes, big hole, and slots respectively; while DC1, DC2, and DC3 refer to the absolute deviation from the nominal sizes for the three characteristics, respectively. The output results of MINITAB, exhibited in Table 1, include the results of the small circle, the big circle, and the slot (either the two slots or the two small circles gave almost identical results). 5. Interpreting statistical results To interpret the results, consider the first part containing the analysis of the small circle. Regression analysis of C1 INDEX versus the absolute deviation from the nominal size of the circle diameter (DC1) resulted in the fitting regression equation: C1 INDEX ¼ 1:0 0:0136DC1
ð2Þ
The equation implies that with each additional unit increase in the deviation from the nominal size, the average index value decreases by 0.0136. The least square estimators (LSEs) of regression coefficients (the constant b0 and the slope b1) are b0 = 0.9996 and b1 = 0.0136. Their standard errors are SE(b0) = 0.000063 and SE(b1) = 0.000155. The t ratios for the constant and the slope are 0.9996/0.000063 = 15781.33 and 0.0136/0.000155 = 87.69, respectively as exhibited in the column labelled T. The last column labelled ‘‘p’’ contains the probability value of the regression coefficients. The t ratio for the slope b1 is 87.69 leads to a very small probability value. Under the null hypothesis of no relationship (b1 = 0), there is almost no chance to get such an extreme value. Hence, one reject – very soundly – the null hypothesis b1 = 0. In other
Table 2
Lower limit of inspection indices.
Quality characteristic
The small hole (circle 1)
The big circle (circle 2)
The slot (slot 1)
Index lower limit
0.993
0.983
0.989
Table 3
83
words, there is a very strong, and negative, association among the C1 INDEX and the absolute deviation of the small circle diameter from its nominal size. The sum of squares, the degree of freedom (240 for total, 239 for error, and 1 for regression), and the mean squares are shown in the ANOVA part. The R2 from the regression is 97%. This value can be obtained by dividing the regression SSR = 0.003114 by the total sum of squares SST = 0.0032106. It says that 97% of the variation in average C1 INDEX can be explained through the linear association of DC1, reflecting a high association of C1 INDEX and DC1. Another interpretation of ‘‘model fit’’ focuses on standard deviations. The standard deviation of the C1 INDEX, not keeping track of deviation DC1, is given by sy = [SST/ (n 1)]1/2 = 0.00366. After factoring in (or adjusting the analysis for) DC1, the standard deviation of the yet unexplained deviations is given by s = [SSE/(n 2)]1/2 = 0.000636. This is the square root of MSE. The reduction from sy = 0.000636 to s = 0.000636 is only 5.75%. The last column of the ANOVA contains the F ratio, F = SSR/(1/MSE) = 7689.67. It serves as a test of the null hypothesis b1 = 0. The probability value to the right of this number is the probability that an F(1, 239) random variable exceeds this value. The probability is virtually zero, implying a solid rejection of the null hypothesis. Note that the F-test in the simple linear regression model is equivalent to the test that looks at the t ratio. The square of the t ratio, (87.69)2 = 7689.6, is identical to the F ratio. In summary, there is a strong association between the C1 INDEX and the DC1; and the linear regression is a good fitting model for this association. The same interpretations can be driven for the results of other quality characteristics. Applying the regression equations in Table 1, the lower limit values for the indices can be found for the assumed tolerance bands. Table 2 shows the lower limit values of the indices for different quality characteristics. It should be noticed that the indices values in Table 2 are valid only for the masks of Fig. 2 and for the assumed tolerance bands. Changes of the masks will result in changes of the number of pixels and, in turn, different values of indices. The developed approach with the resulted indices is then applied to inspect further units of the product. Table 3 includes the resulted indices of the quality characteristics for a sample of those units. As shown, some of them are within tolerance (accepted) and others are out of tolerance (rejected). The shaded values in Table 3 are those corresponding to the nonconforming dimensions. The results realized completely comply with the desired ones.
A sample of the inspected products and their resulted indices.
Circle 1
Circle 2
Circle 3
Slot 1
Slot 2
Acc/Rej
Diameter
Index
Diameter
Index
Diameter
Index
Width
Index
Width
Index
4.2 4.7 4.1 3.8 4.0
0.9981 0.9901 0.9986 0.9959 1
20.0 20.0 20.3 20.5 20.0
1 1 0.9902 0.9856 0.9995
4.0 4.5 4.2 3.4 4.0
1 0.9987 0.9981 0.9867 0.9998
4.3 3.4 3.9 4.2 4.0
0.9938 0.9841 0.9967 0.9984 0.9997
3.9 4.1 4.0 3.8 4.8
0.9966 0.9986 0.9998 0.9953 0.9841
Accept Reject Accept Reject Reject
84
M.H. Hassan, S.L. Diab
6. Conclusions A new visual inspection approach is introduced based on image analysis. The approach is used to test multiple dimensions (quality characteristics) simultaneously; where an index is determined for each quality characteristic. Statistical analysis shows that a regression model is strongly suitable to model the association of the index with the deviation from the nominal size with very small fitting errors. The model is validated by inspecting additional units with known dimensions; and it gave satisfactory results. References [1] Barelli L, Bidini G, Mariani F, Svanziroli M. Neuro-fuzzy network for the classification of buried pipe defects. Eng Appl Artif Intell 2008;21:1065–72. [2] Brzakovic D, Vujovic D. Designing defect classification system: a case study. Pattern Recognit 1996;29. [3] Chen WC, Hsu SW. A neural-network approach for an automatic LED inspection system. Expert Syst Appl 2007;33:531–7. [4] Chiu YP, Lin HD. A hybrid approach based on Hotelling statistics for automated visual inspection of display blemishes in LCD panels. Expert Syst Appl 2009;36. [5] Conners RW, McMillin CW, Lin K, Vasquez-Espinosa RE. Identifying and locating surface defects in wood. IEEE Trans Pattern Anal Mach Intell 1983;5. [6] Mitzias DA, Mertzios BG. A neural multiclassifier system for object recognition in robotic vision applications. Measurement 2004;36:315–30. [7] Fricout G, Jeulin D, Krauth P, Jacquot T. Automatic on-line inspection of non-smooth surface. Wear 2008;264:416–21. [8] Gonzalez RC, Woods RE, Eddins SL. Digital Image Data Processing Using Matlab. Prentice Hall: Pearson Education Inc.; 2004. [9] Guo W, Soibelman L, Garrett Jr JH. Automated defect detection for sewer pipeline inspection and condition assessment. Automat Constr 2009;18. [10] Haralick RM, Shanmugam K, Dinstein J. Textural features for image classification. IEEE Trans Syst Man Cybernet 1973;3:610–21. [11] Mohammed HH, Safaa DL. New approaches for on-line visual inspection of products with multiple characteristics and known tolerances. Int J Rapid Manufacturing 2010:1. [12] Lin HD. Automated defect inspection of light-emitting diode chips using neural network and statistical approaches. Expert Syst Appl 2009;36:219–26. [13] Lin HD. Automated visual inspection of ripple defects using wavelet characteristic based multivariate statistical approach. Image Vision Comput 2007;25:1785–801. [14] Iivarinen J. Surface defect detection with histogram-based texture features. Proc SPIE 2000;4197. [15] Fuertes JJ, Domı´ nguez M, Reguera P, Prada MA, Dı´ azb I, Cuadrado AA. Visual dynamic model based on self-organizing
[16]
[17] [18]
[19] [20] [21]
[22]
[23] [24]
[25]
[26]
[27] [28]
[29]
maps for supervision and fault detection in industrial processes. Eng Appl Artif Intell 2010;23:8–17. Lee S, Vachtsevanos G. An application of rough set theory to defect detection of automotive glass. Proc Institut Mech Eng B J Eng Manuf 2002;216(4):637–41. Leopold J, Gunther H, Leopold R. New developments in fast 3Dsurface quality control. Measurement 2003;33:179–87. Lin HD. Tiny surface defect inspection of electronic passive components using discrete cosine transform decomposition and cumulative sum techniques. Image Vision Comput 2008;26(5):603–21. Mak KL, Peng P, Yiu KFC. Fabric defect detection using morphological filters. Image Vision Comput 2009;27:1585–92. Maldonado JO, Gran˜a M. Recycled paper visual indexing for quality control. Expert Syst Appl 2009;36:8807–15. Olsson J, Gruber S, Web process inspection using neural classification of scattering light. In: Proceedings of the IEEE international conference on industrial electronics, control, instrumentation and automation (IECON’92); 1992. Ramana KV, Ramamoorthy B. Statistical methods to compare the texture features of machined surfaces. Pattern Recognit 1996;29. Siew LH, Hogdson RM. Texture measures for carpet wear assessment. IEEE Trans Pattern Anal Mach Intell 1988;10. Sinha SK, Fieguth PW. A non-conventional quality control system to detect surface faults in mechanical front seals. Automat Constr 2006;15:73–83. Sylla C. Experimental investigation of human and machine-vision arrangements in inspection tasks. Control Eng Pract 2002;10:347–61. Tam SM, Cheung KC. An intelligent sensor for machined parts inspection. Proc Institut Mech Eng B J Eng Manuf 2000;214(8):723–34. Tsai DM, Huang TY. Automated surface inspection for statistical textures. Image Vision Comput 2003;21:307–23. Wilder J. Finding and evaluating defects in glass. In: Freeman H, editor. Machine vision for inspection and measurement. New York: Academic Press; 1989. Liu Y-H, Lin S-H, Hsueh Y-L, Lee M-J. Automatic target defect identification for TFT-LCD array process inspection using kernel FCM-based fuzzy SVDD ensemble. Expert Syst Appl 2009;36:1978–98. Dr. Mohammed H. Hassan is an associate professor in the Faculty of Engineering, Helwan University. He works as consultant in many private and public agents. His research interests include quality management applications, lean manufacturing, project management, and supply chains.
Ain Shams University
Ain Shams Engineering Journal www.elsevier.com/locate/asej www.sciencedirect.com
MECHANICAL ENGINEERING
Visual inspection of products with geometrical quality characteristics of known tolerances Mohammed H. Hassan a b
a,*
, Safaa L. Diab
b
Helwan University, Faculty of Engineering, Helwan, Cairo, Egypt Helwan University, Faculty of Science, Helwan, Cairo, Egypt
Received 24 April 2010; accepted 30 June 2010 Available online 5 November 2010
KEYWORDS Visual inspection; Image processing; Thresholding; Tolerance; Regression; ANOVA
Abstract Image analysis techniques are being increasingly used to automate industrial inspection. The manual activity of inspection could be subjective and highly dependent on the experience of human personnel. In a previous work, the authors presented two approaches utilizing image analysis to inspect products visually. The product is accepted or rejected based on its conformance to specified tolerances, where conformance is analogy measured through statistical indices like correlation and root mean square error. In this work, the authors introduce a novel visual inspection approach that can be used on line to test simultaneously multiple quality characteristics. The approach utilizes image processing tools to deal with the product image; and extract features of its geometrical characteristics. Based on tolerance bands of each characteristic, an index is experimentally developed to reflect the deviation of a quality characteristic dimension from its nominal value; and one can decide whether a characteristic complies with the pre-specified tolerance. Statistical analysis proved that there is a strong association between the developed indices and the deviations of quality characteristics from their target values. Linear regression models are proved to model these associations; and are used to give the corresponding indices’ values relative to the tolerance specifications. The developed approach is
* Corresponding author. E-mail addresses: [email protected] Hassan), [email protected] (S.L. Diab).
(M.H.
2090-4479 Ó 2010 Ain Shams University. Production and hosting by Elsevier B.V. All rights reserved. Peer review under responsibility of Ain Shams University. doi:10.1016/j.asej.2010.09.011
Production and hosting by Elsevier
80
M.H. Hassan, S.L. Diab proved to give a good performance experimentally in detecting non-conforming products; and, in specific, the defect location(s). Ó 2010 Ain Shams University. Production and hosting by Elsevier B.V. All rights reserved.
1. Introduction The rapidly increase in automating production processes should be sustained by automated inspection technologies. Such technologies can help reducing traditional quality control costs and errors; and in turn, eliminating wastes. Inspection is defined as the process of determining whether products deviate from a given set of specifications [13,26]. Manufacturing has long recognized that 100% inspection using a human inspector does not find 100% of the defective items. Since this situation was recognized, inspection has been altered from 100% inspection to sampling inspection using statistical quality control (SQC) which was later incorporated more directly into the production processes using statistical process control (SPC). While well implemented SPC and SQC programs can drastically reduce the fraction of defective products reaching the customer, they could never be perfect for all intended purposes [25]. Therefore, developing an automated visual inspection system will significantly contribute to the quality improvement of products and reduce chances for passing defected parts to next production phases. Automated visual inspection determines the properties of products using visual information and is most often automated by employing machine vision techniques. With recent advances in computer technology, modern manufacturers have turned their attention to machine-vision inspection systems. Such modern devices now inspect metal products [1,15], textile fabrics [19], pipeline [9,24]; and are used in trash separation processes [20], and robotics [6] even while products are on conveyors. In a previous work, the authors [11] introduced statistically based visual approaches that can be used on line to inspect products with multiple dimensions and known tolerances. The approaches’ output is that the inspected product is accepted or rejected based on a calculated statistical index(s). The approaches cannot recognize the location of defect(s) or which quality characteristic causes rejection. In this research, the developed approach integrates image processing approaches with statistical regression to find out a new index for each quality characteristic of the product; and accordingly, determine whether that characteristic conforms to tolerance specifications. 2. Previous work Most of the defect detection systems are focused on non-textured surfaces such as glass panel [28], sheet steel [21] and uniform web materials [2] using simple thresholding or edge detection techniques. Defects in these images can be easily detected because they commonly have distinctly measured values with respect to those of the uniform background. Automatic visual inspection techniques for textured images generally compute a set of textural features in the spatial domain or in the spectral domain, and then search for significant
local deviations in the feature values using various classifiers. In spatial-domain approaches, the commonly used features are the second-order statistics derived from spatial gray-level co-occurrence matrices [10]. They have been applied to wood inspection [5], carpet wear assessment [23], roughness measurement of machined surfaces [22], and surface defect detection [14]. Inspection of dimensional defects has become a critical task for manufacturers who strive to improve product quality and production efficiency [16,18]. Research concerning automated visual inspection grows rapidly. Leopold et al. [17] introduced new approaches in fast 3D-surface quality control. They made a good classification of these approaches and their application fields. Most of researches integrated AI approaches with image processing tools. Lin [12] developed an approach that utilizes neural network and statistical approach to inspect light-emitting diode chips. Fricout et al. [7] developed an on-line approach to inspect the smoothness of metal surfaces. The measurements were based on topographical maps obtained through interferometric microscopy. The resulting data were analyzed by an algorithm based on morphological and statistical features extraction from surface topography, factorial analysis, bootstrap over-sampling and Bayesian classification. Lin [13] presented a wavelet characteristic based approach for the automated visual inspection of ripple defects in the surface barrier layer chips of ceramic capacitors. Among the useful applications of visual inspection is the colour unevenness testing like in LCD monitors which is not easy to be made by humans as a result of human subjectivity and eye fatigue. Chiu and Lin [4] developed a hybrid approach to test blemishes in LCD panels. Their approach is based on Hotelling statistics and image analysis. Other examples includes [1,3,27,29]. The decision of all developed approaches is either to accept or reject the product without determining the source of defect(s). The approach introduced in this research utilizes image processing and statistical approaches to inspect simultaneously multiple quality characteristics. It is recognized by giving a decision for each quality characteristic such that a product defect(s) are recognized; and not just accept/reject decision. 3. The developed approach When an image is digitally transformed and acquired by a computer, it is represented by a two-dimensional matrix. The matrix elements values depend on the type of the image that can be classified as binary, colour, gray scale, etc. As by Gonzalez et al. [8], an image may be defined as a two dimensional function, f(x, y), where x and y are spatial coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of the image. The first step in this approach is to specify non-overlapping regions (called masks) over the image such that each region covers completely one of the image’s quality characteristics.
Visual inspection of products with geometrical quality characteristics of known tolerances The locations of these masks must be determined such that each quality characteristic belongs to one (and only one) mask. A good selection of masks is realized when the mask region can be easily determined its coordinates and the size of the mask is as small as possible. This is considered as an initial selection of masks. The next step is to threshold the image and transform its matrix elements into binary values (0, 1). Final coordinates of masks are specified based on the clusters of zeros and ones that determine the locations of quality characteristics in the image matrix. A final step is to calculate an index for each quality characteristic that reflects the extent of conformance of that quality characteristic to the target or nominal characteristic of the reference image (the corresponding characteristic with nominal dimensions). The index is calculated for each mask (m) using the following equation: Im ¼
nm X Si n m i¼1
m ¼ 1; 2; . . . ; M
where Si is a (0, 1) variable; which is one when the mask’s i pixel is similar to the corresponding one of the reference image and zero otherwise, nm is the number of pixels of mask m, and M represents the number of masks, or number of quality characteristics. The M indices are calculated simultaneously. Each pixel of a product’s image matrix is compared to its corresponding one of the reference image; and according to its location (the mask it belongs to), the index value of that mask is updated. The value of an index is exactly equal one (theoretically) if the quality characteristic dimensions are the same as the reference one and
ð1Þ
Figure 3
Figure 1
Circle one diameter vs. index.
The reference product image.
Figure 4
Figure 2
81
The reference image with masks.
Figure 5
Slot width vs. index.
Circle two diameter vs. index.
82 Table 1
M.H. Hassan, S.L. Diab MINITAB results of the data statistical analyses.
Regression analysis: C1 INDEX versus DC1 The regression equation is C1 INDEX = 1.00 0.0136DC1 Predictor Coef SE Coef Constant 0.999580 DC1 0.0136063 S = 0.0006363, R2 = 97.0%, R2(adj) = 97.0%
0.000063 0.0001552
T
P
15,781.33 87.69
0.000 0.000
Analysis of variance Source
DF
SS
MS
F
P
Regression Residual error Total
1 239 240
0.0031139 0.0000968 0.0032106
0.0031139 0.0000004
7689.67
0.000
Regression analysis: C2 INDEX versus DC2 The regression equation is C2 INDEX = 1.00 0.0347DC2 Predictor Coef SE Coef Constant 0.999663 DC2 0.0347327 S = 0.001384, R2 = 96.4%, R2(adj) = 96.4%
0.000096 0.0004352
T
P
10,412.70 79.82
0.000 0.000
Analysis of variance Source
DF
SS
MS
F
P
Regression Residual error Total
1 239 240
0.012208 0.000458 0.012666
0.012208 0.000002
6370.56
0.000
T
P
6388.84 51.79
0.000 0.000
Regression analysis: S1 INDEX versus DC3 The regression equation is S1 INDEX = 1.00 0.0207DC3 Predictor Coef SE Coef Constant 0.999557 DC3 0.0206534 S = 0.001608, R2 = 91.8%, R2(adj) = 91.8%
0.000156 0.0003988
Analysis of variance Source
DF
SS
MS
F
P
Regression Residual error Total
1 239 240
0.0069354 0.0006180 0.0075534
0.0069354 0.0000026
2682.21
0.000
has no shift in position. Deviation from that target downsizes the index value. Experimentally, one can determine an acceptable value lower limit for Im when the dimensions of the quality characteristic fall within the tolerance region. The approach can also be used to detect a shift in the quality characteristic position. If, for example, a hole’s centre is shifted from its correct position, the calculated index will go down even if the hole diameter is within tolerance. Calculating the acceptable Im value for different characteristics, a product can be determined whether there is a problem(s) with one or more of its quality characteristics one by one. 4. Application The developed approach is applied to the product exhibited in Fig. 1. As shown, the product has five quality characteristics which are the two small holes (4 mm diameter), the two slots (4 mm width), and the middle big hole (20 mm diameter). All dimensions of the quality characteristics assumed to have a tolerance of ±0.5 mm.
At first, the five masks are selected as shown in Fig. 2; where each mask dominates its quality characteristic. The image is then thresholded and transformed to a binary image. The final coordinates of the masks are determined based on the matrix clusters of zeros and ones. The determination of acceptable indices’ values of the quality characteristics, as well as the digital image data processing requires experimental work. In this context, the authors used 170 images with different dimensions that all fall within the tolerance specifications,70 images with some out of tolerance dimensions, and 30 images with shifted quality characteristic(s) positions. The resulted quality characteristics indices of all tested images are plotted versus the characteristics’ dimensions. Figs. 3–5 exhibit the indices of the small holes, the slots, and the large hole, respectively. As the selected masks of similar characteristics (the two small holes or the two slots) are identical, the resulted indices of these characteristics are almost the same. Accordingly, the further analyses will be concerned with three characteristics: the small hole, the big hole, and the slot. It can be concluded that one can find an index value (lower limit) for each quality characteristic that tells whether that
Visual inspection of products with geometrical quality characteristics of known tolerances characteristic conforms to specifications. This can be achieved by plotting the specification limits, as in Fig. 3, and find the corresponding index value. The figures show a high similarity of the index values around the nominal sizes. Linear regression analyses are conducted to test the relationships between the indices and the deviation from the nominal sizes of the different quality characteristics. The analyses gave a good fitting regression equations with very low fitting errors. In Table 1, the MINITAB results output for the regression analyses and the corresponding ANOVA results are exhibited. C1, C2 and S1 in Table 1 refer to the small holes, big hole, and slots respectively; while DC1, DC2, and DC3 refer to the absolute deviation from the nominal sizes for the three characteristics, respectively. The output results of MINITAB, exhibited in Table 1, include the results of the small circle, the big circle, and the slot (either the two slots or the two small circles gave almost identical results). 5. Interpreting statistical results To interpret the results, consider the first part containing the analysis of the small circle. Regression analysis of C1 INDEX versus the absolute deviation from the nominal size of the circle diameter (DC1) resulted in the fitting regression equation: C1 INDEX ¼ 1:0 0:0136DC1
ð2Þ
The equation implies that with each additional unit increase in the deviation from the nominal size, the average index value decreases by 0.0136. The least square estimators (LSEs) of regression coefficients (the constant b0 and the slope b1) are b0 = 0.9996 and b1 = 0.0136. Their standard errors are SE(b0) = 0.000063 and SE(b1) = 0.000155. The t ratios for the constant and the slope are 0.9996/0.000063 = 15781.33 and 0.0136/0.000155 = 87.69, respectively as exhibited in the column labelled T. The last column labelled ‘‘p’’ contains the probability value of the regression coefficients. The t ratio for the slope b1 is 87.69 leads to a very small probability value. Under the null hypothesis of no relationship (b1 = 0), there is almost no chance to get such an extreme value. Hence, one reject – very soundly – the null hypothesis b1 = 0. In other
Table 2
Lower limit of inspection indices.
Quality characteristic
The small hole (circle 1)
The big circle (circle 2)
The slot (slot 1)
Index lower limit
0.993
0.983
0.989
Table 3
83
words, there is a very strong, and negative, association among the C1 INDEX and the absolute deviation of the small circle diameter from its nominal size. The sum of squares, the degree of freedom (240 for total, 239 for error, and 1 for regression), and the mean squares are shown in the ANOVA part. The R2 from the regression is 97%. This value can be obtained by dividing the regression SSR = 0.003114 by the total sum of squares SST = 0.0032106. It says that 97% of the variation in average C1 INDEX can be explained through the linear association of DC1, reflecting a high association of C1 INDEX and DC1. Another interpretation of ‘‘model fit’’ focuses on standard deviations. The standard deviation of the C1 INDEX, not keeping track of deviation DC1, is given by sy = [SST/ (n 1)]1/2 = 0.00366. After factoring in (or adjusting the analysis for) DC1, the standard deviation of the yet unexplained deviations is given by s = [SSE/(n 2)]1/2 = 0.000636. This is the square root of MSE. The reduction from sy = 0.000636 to s = 0.000636 is only 5.75%. The last column of the ANOVA contains the F ratio, F = SSR/(1/MSE) = 7689.67. It serves as a test of the null hypothesis b1 = 0. The probability value to the right of this number is the probability that an F(1, 239) random variable exceeds this value. The probability is virtually zero, implying a solid rejection of the null hypothesis. Note that the F-test in the simple linear regression model is equivalent to the test that looks at the t ratio. The square of the t ratio, (87.69)2 = 7689.6, is identical to the F ratio. In summary, there is a strong association between the C1 INDEX and the DC1; and the linear regression is a good fitting model for this association. The same interpretations can be driven for the results of other quality characteristics. Applying the regression equations in Table 1, the lower limit values for the indices can be found for the assumed tolerance bands. Table 2 shows the lower limit values of the indices for different quality characteristics. It should be noticed that the indices values in Table 2 are valid only for the masks of Fig. 2 and for the assumed tolerance bands. Changes of the masks will result in changes of the number of pixels and, in turn, different values of indices. The developed approach with the resulted indices is then applied to inspect further units of the product. Table 3 includes the resulted indices of the quality characteristics for a sample of those units. As shown, some of them are within tolerance (accepted) and others are out of tolerance (rejected). The shaded values in Table 3 are those corresponding to the nonconforming dimensions. The results realized completely comply with the desired ones.
A sample of the inspected products and their resulted indices.
Circle 1
Circle 2
Circle 3
Slot 1
Slot 2
Acc/Rej
Diameter
Index
Diameter
Index
Diameter
Index
Width
Index
Width
Index
4.2 4.7 4.1 3.8 4.0
0.9981 0.9901 0.9986 0.9959 1
20.0 20.0 20.3 20.5 20.0
1 1 0.9902 0.9856 0.9995
4.0 4.5 4.2 3.4 4.0
1 0.9987 0.9981 0.9867 0.9998
4.3 3.4 3.9 4.2 4.0
0.9938 0.9841 0.9967 0.9984 0.9997
3.9 4.1 4.0 3.8 4.8
0.9966 0.9986 0.9998 0.9953 0.9841
Accept Reject Accept Reject Reject
84
M.H. Hassan, S.L. Diab
6. Conclusions A new visual inspection approach is introduced based on image analysis. The approach is used to test multiple dimensions (quality characteristics) simultaneously; where an index is determined for each quality characteristic. Statistical analysis shows that a regression model is strongly suitable to model the association of the index with the deviation from the nominal size with very small fitting errors. The model is validated by inspecting additional units with known dimensions; and it gave satisfactory results. References [1] Barelli L, Bidini G, Mariani F, Svanziroli M. Neuro-fuzzy network for the classification of buried pipe defects. Eng Appl Artif Intell 2008;21:1065–72. [2] Brzakovic D, Vujovic D. Designing defect classification system: a case study. Pattern Recognit 1996;29. [3] Chen WC, Hsu SW. A neural-network approach for an automatic LED inspection system. Expert Syst Appl 2007;33:531–7. [4] Chiu YP, Lin HD. A hybrid approach based on Hotelling statistics for automated visual inspection of display blemishes in LCD panels. Expert Syst Appl 2009;36. [5] Conners RW, McMillin CW, Lin K, Vasquez-Espinosa RE. Identifying and locating surface defects in wood. IEEE Trans Pattern Anal Mach Intell 1983;5. [6] Mitzias DA, Mertzios BG. A neural multiclassifier system for object recognition in robotic vision applications. Measurement 2004;36:315–30. [7] Fricout G, Jeulin D, Krauth P, Jacquot T. Automatic on-line inspection of non-smooth surface. Wear 2008;264:416–21. [8] Gonzalez RC, Woods RE, Eddins SL. Digital Image Data Processing Using Matlab. Prentice Hall: Pearson Education Inc.; 2004. [9] Guo W, Soibelman L, Garrett Jr JH. Automated defect detection for sewer pipeline inspection and condition assessment. Automat Constr 2009;18. [10] Haralick RM, Shanmugam K, Dinstein J. Textural features for image classification. IEEE Trans Syst Man Cybernet 1973;3:610–21. [11] Mohammed HH, Safaa DL. New approaches for on-line visual inspection of products with multiple characteristics and known tolerances. Int J Rapid Manufacturing 2010:1. [12] Lin HD. Automated defect inspection of light-emitting diode chips using neural network and statistical approaches. Expert Syst Appl 2009;36:219–26. [13] Lin HD. Automated visual inspection of ripple defects using wavelet characteristic based multivariate statistical approach. Image Vision Comput 2007;25:1785–801. [14] Iivarinen J. Surface defect detection with histogram-based texture features. Proc SPIE 2000;4197. [15] Fuertes JJ, Domı´ nguez M, Reguera P, Prada MA, Dı´ azb I, Cuadrado AA. Visual dynamic model based on self-organizing
[16]
[17] [18]
[19] [20] [21]
[22]
[23] [24]
[25]
[26]
[27] [28]
[29]
maps for supervision and fault detection in industrial processes. Eng Appl Artif Intell 2010;23:8–17. Lee S, Vachtsevanos G. An application of rough set theory to defect detection of automotive glass. Proc Institut Mech Eng B J Eng Manuf 2002;216(4):637–41. Leopold J, Gunther H, Leopold R. New developments in fast 3Dsurface quality control. Measurement 2003;33:179–87. Lin HD. Tiny surface defect inspection of electronic passive components using discrete cosine transform decomposition and cumulative sum techniques. Image Vision Comput 2008;26(5):603–21. Mak KL, Peng P, Yiu KFC. Fabric defect detection using morphological filters. Image Vision Comput 2009;27:1585–92. Maldonado JO, Gran˜a M. Recycled paper visual indexing for quality control. Expert Syst Appl 2009;36:8807–15. Olsson J, Gruber S, Web process inspection using neural classification of scattering light. In: Proceedings of the IEEE international conference on industrial electronics, control, instrumentation and automation (IECON’92); 1992. Ramana KV, Ramamoorthy B. Statistical methods to compare the texture features of machined surfaces. Pattern Recognit 1996;29. Siew LH, Hogdson RM. Texture measures for carpet wear assessment. IEEE Trans Pattern Anal Mach Intell 1988;10. Sinha SK, Fieguth PW. A non-conventional quality control system to detect surface faults in mechanical front seals. Automat Constr 2006;15:73–83. Sylla C. Experimental investigation of human and machine-vision arrangements in inspection tasks. Control Eng Pract 2002;10:347–61. Tam SM, Cheung KC. An intelligent sensor for machined parts inspection. Proc Institut Mech Eng B J Eng Manuf 2000;214(8):723–34. Tsai DM, Huang TY. Automated surface inspection for statistical textures. Image Vision Comput 2003;21:307–23. Wilder J. Finding and evaluating defects in glass. In: Freeman H, editor. Machine vision for inspection and measurement. New York: Academic Press; 1989. Liu Y-H, Lin S-H, Hsueh Y-L, Lee M-J. Automatic target defect identification for TFT-LCD array process inspection using kernel FCM-based fuzzy SVDD ensemble. Expert Syst Appl 2009;36:1978–98. Dr. Mohammed H. Hassan is an associate professor in the Faculty of Engineering, Helwan University. He works as consultant in many private and public agents. His research interests include quality management applications, lean manufacturing, project management, and supply chains.