- Email: [email protected]
An imaging approach for the automatic thresholding of photo defects
Accepted Manuscript
An imaging approach for the automatic thresholding of photo defects Neelam Bhardwaj , Prof. Suneeta Agarwal , Vikash Bhardwaj PII: DOI: Reference:
S0167-8655(15)00112-9 10.1016/j.patrec.2015.03.019 PATREC 6202
To appear in:
Pattern Recognition Letters
Received date: Accepted date:
31 August 2014 23 March 2015
Please cite this article as: Neelam Bhardwaj , Prof. Suneeta Agarwal , Vikash Bhardwaj , An imaging approach for the automatic thresholding of photo defects, Pattern Recognition Letters (2015), doi: 10.1016/j.patrec.2015.03.019
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
1
Highlights
Evaluation on limitations of valley emphasis method for thresholding of photo defects. Proposed technique of photo defect detection involves imaging of defected photos by sensing the light after passing through them instead of sensing the reflection from their surfaces and then applying valley emphasis method to threshold defect objects.
The proposed approach is able to locate the defects with very small misclassification error without requirement of any defect object features. It overcomes the limitation of thresholding techniques to distinguish the defect objects in presence of similar photo objects and in cases of defects having very low contrast with their back ground (unimodal histogram). The obtained threshold values with proposed approach for all cases of additive defects are less than 50. It reduces the time complexity of defect detection by least 1/5 times as the dynamic range of intensity levels for defected pixels in histogram confine to 0-50 instead of 0-255. Presently the proposed approach is applicable only for physically available photos having light transitivity, but imaging with IR and x-ray may also be explored to overcome the current limitation.
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
An Imaging Approach for the Automatic Thresholding of Photo Defects Neelam Bhardwaj1a, Prof. Suneeta Agarwal 2b, Vikash Bhardwaj 3c a
PhD scholar, Computer Science, MNNIT, Allahabad, UP, INDIA Prof. (Dr.) Suneeta Agarwal, Computer Science, MNNIT, Allahabad, UP, INDIA c Scientist, ADRDE, Agra, UP, INDIA b
CR IP T
ABSTRACT
AN US
Automatic thresholding of photo defects means to accurately locate defect objects. The available approaches for automatic thresholding determine the optimal threshold values and segment the image into objects based on their gray level distribution. For defect object identification in images with multimodal distributions, these techniques also require knowledge of the defect object features, such as shape and size, which limits the applicability of these techniques because the defect object features may vary widely. Additionally, these methods result in extensive misclassification errors in the presence of photo objects similar to defect objects and unimodal distribution. We evaluated the limitations of the valley emphasis method and proposed a new approach that involves the imaging of a defected photo by sensing the light after it passes through photo and then applying the valley emphasis method for thresholding to identify defect objects. The obtained results are better even with the above discussed constraints of the available automatic thresholding approaches. Although the proposed technique is applicable only for physically available objects, it may contribute significantly towards the accuracy of machine vision based applications.
AC
CE
PT
ED
M
Key words: Thresholding, photo defect detection, imaging, valley emphasis method.
Tel.: +91-09411464637; e-mail: [email protected]
ACCEPTED MANUSCRIPT
1. Introduction
2
In the spatial gray level distribution domain, Wanga et al. (2008) attempted to explore a novel global image thresholding method based on the Parzen window estimate of an unknown gray value probability density function rather than the gray level histograms of an image. The Parzen window effectively integrates image histogram information with explicit spatial information about pixels of different gray levels. The MHUE thresholding method ((Saha and Udupa, 2001) utilizes the spatial information of an image. In this method, the spatial information is implicitly represented by region homogeneity. L. Maddalenaa et al. (2009) used a fusion-based detection approach for scratches in corrupted image sequences. They fused algorithms utilizing Radon projection and then detected local maxima using Top Hat transformation and developed a scratch mask by assigning value 1 to all the pixels of columns that contain more than 70% pixels with value 1. Anil C. Kokaram (2004) presented his work on the restoration of defected motion pictures, where he also discussed the Spike Detection Index (SDI), Rank Order Detector (ROD), and image histogram based approaches and morphological approaches for defect detection.
AN US
The basic idea of thresholding is to find an optimal gray-level by analyzing the gray-level distribution and extracting the objects of interest from an image. There are two types of automatic thresholding techniques, global thresholding and local thresholding. In global thresholding, a single threshold value is taken from the histogram of the entire image, and in local thresholding, the local gray-level distributions are explored to determine multiple threshold values. For defect detection applications, global thresholding is easier to implement, but its applicability is limited in photo images with distribution close to unimodal, whereas local thresholding methods work for both nearly unimodal and multimodal distributions, but they are complex.
with a smaller variance. To address this problem, they proposed an improved Otsu algorithm that constrains the search range of the gray levels. The minimum error thresholding (MET) method also assumes that the image can be characterized by a mixture distribution of foreground and background pixels (Kittler and Illingworth, 1986). The entropy-based techniques exploit the entropy of the distribution of the gray levels and interpret the maximization of the entropy of the thresholded image as indicative of maximum information transfer. The maximum cross entropy method (Kapur et al., 1985 and Abutaleb, 1989) considers the foreground and background of an image as two different information sources, so that when the sum of these two class entropies reaches its maximum, the image is said to be optimally thresholded. The Tsallis entropy-based method (Portes de Albuquerque et al., 2004), Renyi’s entropy-based method (Sahoo and Arora, 2004), and supervised Lloyd algorithm (Morii, 1995) are also typical examples of this class.
CR IP T
Photo defect detection is the foremost essential requirement for the development of accurate images. Generally, photos are damaged by the occurrence of defects either by the addition of layers of substance (additive defects) or by the removal of layers of substance from their surfaces. The defect standard taxonomy is available (Ardizzone et al., 2009). Defects alter the intensity values of pixels at their locations and can be identified with the help of optimal intensity thresholding. Automatic thresholding of defects reduces the span of the histogram for analysis to locate defect objects. This enhances the effectiveness of vision-based industrial applications and may facilitate real-time corrective action in the manufacturing phase itself.
M
Because of its scope, which includes other areas of image processing and machine vision based applications, much work on automatic thresholding has already been conducted. Recently, Sezgin and Sankur (2004) conducted an exhaustive survey of non-parametric and statistical approaches for image thresholding, where they categorized the approaches according to the information used, such as the histogram shape, measurement space clustering, entropy, object attributes, spatial correlation, and local gray-level surfaces and compared their performance.
AC
CE
PT
ED
Among clustering-based methods, where the gray level samples are clustered in two parts as the background and foreground, the Otsu method (Otsu, 1979) is the most popular method. This method selects threshold values that maximize the between class variances of the histogram. This technique works well but finds limitations in the case of unimodal or close to unimodal gray-level histograms. Many modifications have been added to the Otsu method to overcome its limitations. Hui Fuang Ng (2006) revised the Otsu method as the valley-emphasis method, in which the valley point information is considered in the objective function. In this method, the optimal threshold is decided by maximizing the between class variance and minimizing the within class variance. In this method, threshold values are located as close as possible to the valley points in the histogram. The modified valley emphasis method (Jiu Lun Fan and Bo Lei, 2012) covers the limitation for the case where the variance of the object is different from that of the background. They considered not only the valley-point information but also the neighborhood gray information around the valley-point and selected a threshold value that has small probabilities in its neighborhood area and also maximizes the between class variance in the gray-level histogram. Hui Fuang Ng et al. (2013) further improved the valley-emphasis method by introducing a Gaussian weighting scheme that efficiently uses the neighborhood information for inclusion in the objective function to enhance the effect of the weight factor. Hou et al. (2006) proposed minimum class variance thresholding (MCVT) and compared it with the Otsu method. They found that in the Otsu method, the resulting threshold is biased towards the component with a larger class variance or larger class probability. Xiangyang Xu et al. (2011) also described that when the within class variances of two classes are different, the Otsu threshold is biased toward the class with a larger variance. As a result, partial pixels belonging to this class will be misclassified into the other class
Chang et al. (2005) worked in a different way to threshold defects due to ink spray and scratches by thick colored pens. They used the low intensity variance property of ink spreading at defect locations and designed a thresholding filter to determine low varying and smooth portions of the histogram. Then, they tracked the decreasing rate of the area of thresholded objects. If the decreasing rate is high, then an object is not likely to be a defect object, and if the reduction in an object's size is less than half of the intensity value reduction of one, it is claimed as a defect object. All the above discussed methods of automatic thresholding only produce the optimal segmentation of a defected input image into its constituent objects but cannot claim that a particular object is a defect object. Still, in the case of bimodal distributions, the defect objects can be filtered based on size. However, for defect detection in the case of a multimodal distribution, the entire image is scanned for comparison with available defect object properties for the best match, which makes these methods more complex and time consuming. The requirement of defect object features related to their shape and size limit the applicability of these techniques, as defect features are random in nature. These methods also generate high misclassification error in the case of similarity of defect objects to their background and undamaged image objects. Automatic thresholding has been widely used in the industry for automated visual inspection of defects (Newman, 1995). If a technique with less time complexity and independent to defect properties is available, it will enhance the effectiveness of visionbased applications and make them able to take real time
ACCEPTED MANUSCRIPT
We have proposed a new technique of photo defect detection that images defected photos by sensing the light that passes through them instead of by sensing the reflection from their surfaces to applying the valley emphasis method to threshold defect objects. Imaging of a defected photo by the proposed technique enhances the contrast of defect objects with the image objects. In fact, a unimodal distribution becomes bimodal or multimodal, which makes them appropriate inputs to undergo automatic thresholding for accurate detection. We have experimented with approximately 20 input photos defected with various types of additive defects. We have also evaluated the limitations of the valley emphasis method and compared the result with the proposed method by taking two cases of suitably defected input photos. The results obtained are even better with above discussed constraints of currently available approaches. Due to the enhanced contrast of defect objects with image objects, this approach is able to distinguish the defect object from similar image objects. Although the proposed technique is only applicable to physically available photos, it may also contribute significantly towards the accuracy of machine vision based applications.
Fig. 2. Experimental setup
AN US
In this paper we have used the term "day image" for an image acquired by scanning the defected photo image by a conventional method i.e., by sensing the light reflection, "night image" for an image acquired by imaging the defected photo with the proposed method i.e., by sensing the light passed through the defected image, and "additive defect" for a defect caused by the addition of defect material.
For experiments, we prepared a setup as shown in figure 2 with the help of a wooden box, a white 5 W CFL as a light source, a 4 mm thick transparent glass as a base for holding the subject photos and 8 megapixel Sony cameras for image acquisition. The vertical distance between the light source and transparent glass/photo is kept fixed. For better results, the imaging is performed in a dark room to minimize the effect of light reflection.
CR IP T
corrective action at the manufacturing phase to reduce waste during inspection.
3
M
In this paper, after an introduction about the problem and related work, Section II introduces the proposed imaging technique, Section III provides details about obtaining the threshold value, Section IV describes the details of the experiments and the contribution made, and the report is finally concluded in Section V.
ED
2. Proposed approach
Having a defected photo physically available as input, the goal is to locate the defect objects and mark them with an appropriate color to highlight their presence. The overall procedure is illustrated in figure 3. During night image acquisition using the experimental set up, camera flash and room lights are kept OFF, while the light source of the imaging setup is kept ON. For day image acquisition, the light source was kept OFF, while the camera flash and room lights were kept ON. The camera position and settings are fixed to obtain both types of the images of the same size and with one to one mapping. After the acquisition of the images, the gray level distributions of the night images are analyzed for the optimal threshold value to locate the defected pixel, and then these detected pixels are highlighted in the corresponding day image for better appearance. The image processing is carried out in the luminous region using MATLAB.
AC
CE
PT
Generally, day images are used for thresholding in defect detection. However, in the proposed approach, the night images are explored primarily and used as inputs for the valley emphasis method for deciding the optimal threshold value. Due to the presence of additive defects, the image thickness at defect locations increases compared to the remaining undamaged regions. This keen observation led us to develop an entirely different approach than that found in the literature, where we have developed a new imaging setup for sensing the light passed through the defected photos instead of the usual way of scanning by sensing the reflected light from their surface. The proposed approach involves night image and day image acquisition followed by analysis and processing. The details about the proposed approach are given in the following subsections.
2.2. Defect detection procedure
Fig. 3. Defect detection procedure
2.1. Proposed Imaging and Experimental setup
3. Automatic thresholding of defects
As the proposed imaging approach is based on sensing the light after passing through the input photos, light is projected from one side and sensed after passing through the subject photos by an image acquisition device placed on the opposite side, as shown in figure 1.
3.1. Theoretical estimates of defect locations
Fig. 1. Proposed imaging approach
The spatial intensity distribution of the light passed through the photos varies directly according to their spatial thickness variation. In the case of additive defects, the surface thickness at the defected locations of the photos becomes more, and in the case of scratches, the surface thickness at the defected area becomes less. In both cases, the thickness of the photos at the defected locations becomes different compared to the remaining undamaged portion of the photos. Due to this effect of a defect on photo thickness, the defect locations boost their contrast in the luminous intensity domain of the night images.
4
ACCEPTED MANUSCRIPT
For example, if the photo thickness at the defected area becomes double or half compared to the undamaged locations, then according to the Beer – Lambert law, the intensity of the night image at the defected locations will be less by a factor of ‘102α’ in the case of additive defects and more by the same factor in the case of scratch defects, where α = absorption coefficient of the material. The night image, defected at location (i, j) in the luminous domain, may be described using equation (1) and equation (2).
( )
( )
( )
( ) ( )
{
where G(i, j) is the night image intensity in the presence of the defect, I(i, j) is the night image intensity at the defected location in the absence of the defect, D(i, j) is the effect of the defect at location (i, j), ‘K’ is the sensitivity of the camera focal plane array and I0 is the intensity of the incident light.
Fig. 4. Black and white undamaged photo. (a) Day image, (b) Night image (c) Relative histograms
In the second case, defect objects are removed manually from the night images. Figure 5 shows that in the first case, the number of pixels on the lower side is higher than in the second case. The sharp rise at the upper side in the histogram in second case is because of the intentional replacement of defect objects with white color. Thus, it is validated that in the night image histograms, only the additive defect objects are confined to the lower most side within the intensity limit of less than 50.
M
AN US
Due to the defects, factor ‘D’ affects the intensity at the defect locations significantly and shifts the defected objects either at the lower side in the case of additive defects or at the upper side in case of scratches in the luminous intensity histogram of the night image. More values of the defect material thickness and its absorption coefficient would increase the effectiveness of factor ‘D’ and produce better results. The undamaged photo objects of dark colors and light colors also shift their locations toward the interior from both sides in the night image histogram compared to their locations in the corresponding day image histograms. This is shown in figure 4. Thus, only the defect objects of the photos are confined at the lower most and uppermost regions in the luminous histogram of the night image. Hence, it is estimated that in the proposed imaging, the additive defect regions are confined within the lower most lobe of the multimodal distribution.
CR IP T
( )
3.2. Validation of theoretical estimate
ED
For the validation of above estimates, we have taken various input photos and compared the night images with the corresponding day images. The outcomes are discussed in the following subsection.
PT
3.2.1. Contraction of night image histogram for undamaged photos
AC
CE
In the previous section, it is theoretically estimated that the undamaged photo objects ranging from dark to light colors also shift their locations toward the interior from both sides of the night image histogram compared to their locations in the corresponding day image histograms. To quantify this, we compared the histograms of the day and night image of a black and white undamaged input photo. A black and white colored photo is preferred because these color pixels are present at the extreme ends of the day image histogram of any multi colored photo. Figure 4 shows that in the night image histogram of same photo, the lowest intensity pixels of the corresponding day image shift by a factor of approximately 30 away from origin, and the highest intensity pixels shift towards the origin by a factor of approximately 10. Thus, the undamaged photo objects of dark colors and light colors also shift their locations toward interior from both sides in night image histogram compared to their locations in the corresponding day image histograms. 3.2.2. Effect of defect objects on the night image histogram To validate that only additive defect objects of photos are confined at the lowermost side of the luminous histogram of night image, we compared two cases of night image histograms of photos with additive defect objects. One case includes the defect objects, and other case does not include the defect objects.
3.3. Threshold value
With the above conclusion that only additive defect objects are present on the lower side of the night image histogram, we take the difference between the night and day image histograms as an input to the valley emphasis method and also imposed the intensity limit of a maximum of 50. The difference between the histograms is taken because it has clear valley points, and only the additive defect objects are emphasized in the lowermost lobe. The basis of the valley-emphasis method is to select a threshold value that has a small probability of occurrence, i.e., a valley in the gray-level histogram, and also maximize the between group variance, as in the Otsu method. The detail of the formulation of the proposed method is described below. The gray level histograms of the night image and day images can be represented by the intensity functions f n(i) and fd(i), where i ranges from 0 to L 1, and L is the number of distinct graylevels. Then, the difference function f(i) is taken as the input to the Valley method and defined as ()
* ()
( )+
( )
If the number of pixels at the ith gray level in f(i) is represented by ni and n is the total number of pixels in ( ), then the probability of the occurrence of the ith gray-level is defined as⁄
( )
The average gray-level of ( ) is defined as: ∑
( )
5
ACCEPTED MANUSCRIPT
Fig. 5. Night image histograms with the presence and absence of additive defects. a, b & c correspond to the test images of figs. 10, 11 & 12, respectively
|
The single thresholding means that the image pixels are grouped into two classes separated by a threshold value,‘t’.
()
()
∑
( )
∑
and the mean gray-level values of these classes can be computed as ∑
⁄
()
()
∑
⁄
() ( )
Using discriminate analysis, Otsu showed that the optimal threshold t can be determined by maximizing the between-class variance, i.e., ()
{
()
()
( )}
( )
()
()
()
( ))} ( )
ED
)(
M
Later, Valley modified equation (8) for the optimal threshold ) as given in value calculation by adding a weight factor ( equation (9): {(
|
|
| |
Where FO denotes the foreground area pixels or the manually thresholded defect object area (pixels) using Adobe Photoshop software, and BO denotes the background area pixels or the rest of the image area other than the defect pixels. FT denotes the detected pixels as defected pixels, and BT denotes the unmarked defected pixels using the proposed approach. Basically, this misclassification error reflects the percentage of background pixels wrongly assigned to the foreground, and conversely, foreground pixels wrongly assigned to the background. It may range from zero for no error, to one for completely incorrect detection. For the evaluation of each case of defect detection,
first we obtained the optimal threshold value using the proposed approach and then applied this threshold value to the corresponding night images containing the defect objects and again applying to the same night image after manually removing defect objects from it using software such as Adobe Photoshop to identify false detection and false rejection. The experimental results are discussed as three cases of input images to analyze the efficacy of proposed approach compared to the Valley emphasis method.
AN US
()
|
CR IP T
Then, the probabilities of these two classes are:
|
|
The proposed approach is able to identify and locate defect objects having very little contrast with their background. The day images of these types of defected photos have a unimodal gray level distribution, whereas their corresponding night images have at least bimodal gray level distribution. The outcomes of such examples are shown in figures 6 and 7. The misclassification errors are calculated after applying the valley emphasis method on day image histograms and on the difference between night and day image histograms, as given in table 1. It is observed that the proposed approach, in this case, is able to locate defected pixels
AC
CE
PT
The key of the valley-emphasis formulation is the weight ). is the probability of the occurrence of the tth grayfactor( level. The smaller the value is, the larger the weight value is. This weight ensures that the resulting threshold always resides at the valley of the gray-level distribution. The proposed approach limits the value of t in equation (9) to be less than 50 and emphasizes only the first effective valley point as a threshold value. Thus, the number of iterations to find the threshold value is at least 5 times less compared to the existing approach of the Valley method. The modified expression is given in equation (10):
4.1. Case1: Defected photo with unimodal gray level distribution
{(
)(
()
()
()
( ))} (
)
The optimal threshold values obtained with the proposed approach and the corresponding misclassification errors for the defected input photos are given in tables 1, 2 and 3 in the results section of this paper. 4. Results and Contributions Approximately 20 defected photos on a white sheet as a base and defected with various additive defects were collected as input. Encouraging results for defect detection were obtained. We evaluated the performance of the proposed method by calculating the misclassification errors using the definition (Yasnoff et al., 1977) as-
Fig. 6. Photo where defect objects have low contrast with background. a. Day image, b. Night image, c & d. Histograms with threshold marked, e & f. Result.
ACCEPTED MANUSCRIPT
6
Table 1. Results for applying valley emphasis method on day and night images of defected photo with unimodal histogram
Photo
Size
Size of defects
Figure 6 Figure 7
233562 258640
20551 55042
Valley Threshold value Day Night image image 31 106 17 103
Detected defect pixels False detection Day image 231773 258351
Night image 20699 55138
Night image 150 134
4.3. Case 3: Defected photos with Multimodal gray level distribution We have tested the proposed scheme on more than 20 multi colored and multi object photos with additive defects. Some of the results are shown in figures 10, 11 and 12. The misclassification errors in these cases are given in table 3. The proposed approach is able to threshold the defect objects from the remaining undamaged portion and obtain the threshold value of less than 50. If we apply the Valley method on day images of
CR IP T
with very small misclassification error, whereas the Valley emphasis method, if applied on a day image gives very high misclassification error, and in fact categorizes defect objects and background as a single object, as shown in figure 6.e and 7.e marked with a white color.
Day image 210276 202516
Misclassification error Day Night image image 0.8963 0.00065 0.7799 0.00066
AN US
the subjected photos with multimodal distribution, only the segmented components will be obtained, but it is not possible to declare any of the particular classes of objects as defects without prior knowledge of the defect.
Fig. 7. Photo where defect objects have low contrast with background. a. Day image, b. Night image, c & d. Histograms with threshold marked, e & f.
M
4.2. Case2: Defected photos with multimodal gray level distribution where defect objects have similarity with undamaged photo objects
Fig. 9. Photo with defect object similar to images objects in terms of color. a. Day image, b. Night image, c & d. Histograms with threshold marked, e & f. Result images.
AC
CE
PT
ED
We have generated the test images as shown in figure 8.a and 9.a. In the first test case, the blue colored circular object at the center is intentionally placed as a defect object, which is similar to the objects at the corners of the original photo, and in the second case, a multicolored photo is defected with a black color that is almost similar to the background at its location. The results of this case are given in table 2. Although the day images of these types of defected photos have a multimodal gray level distribution, the application of the valley emphasis method on the day images of these photos categorize the defect objects in same class as similar image objects, whereas the proposed approach in such cases is able to locate and distinguish the defected pixels from similar undamaged image pixels with very small misclassification error. The outcomes of these examples are shown in the result images of figures 8 and 9.
Fig. 8. Photo having defect object similar to images objects. a. Day image, b. Night image, c & d. histograms with threshold marked, e & f. result images.
Fig. 10 .Gray colored photo defected with black and white color. a. Day images, b. Night images, c. Result image d. Histograms.
ACCEPTED MANUSCRIPT
7
Table 2. Results for applying valley emphasis method on day and night images of defected photo, where defect object has similarity to undamaged photo objects Valley Threshold Misclassification Detected defect pixels False detection value error Size of Photo Size defects Day Night Day Night Day Night Day Night image image image image image image image image Figure 8 3273450 75182 89 23 254030 65313 186468 0 0.0593 0.0030 Figure 9 303094 7267 101 12 211952 8019 203324 3026 0.6663 0.0175 Table 3. Results for applying valley emphasis method on night images of defected photo with multimodal histogram Size of Valley Threshold Detected defect Misclassification Photo Size False detection defects value pixels error Figure 10 782738 64571 20 64841 281 0.000373 Figure 11 1593584 99492 35 99403 45 0.000143 Figure 12 1857136 130802 36 136464 5666 0.0031
4.4. Contributions
AN US
CR IP T
The proposed approach is able to locate the defects with very small misclassification error without the requirement of any defect object features. It overcomes the limitation to distinguish the defect objects in the presence of similar photo objects and in cases of defects having very low contrast with their back ground. The obtained threshold values for all cases are less than 50, which reduces the time consumption of defect detection by least 1/5 times, as the dynamic range of intensity levels for the defected pixels in the histogram are confined to 0-50 instead of 0255. Thus, the proposed approach will enhance the performance of machine vision based applications.
ED
M
5. Conclusion and future scope
AC
CE
PT
Fig. 11. Multi colored defected photo. a. Day images, b. Night images, c. Result image d. Histograms.
The results obtained and presented above are encouraging. The performance of the proposed approach is better even with the current limitations of thresholding approaches. The approach proposed in this paper is also able to distinguish the defect objects from similar image objects. The proposed defect detection consumes less time and prompts its applicability in vision based industrial applications towards an automated correction mechanism during the production phase itself to save waste during the later inspection stage. The work to apply the proposed approach towards scratch detection is also in progress. In the future, imaging with light in the IR and x-ray regions may also be explored to overcome the current limitation of the proposed method i.e., photos with non-light transmitting bases. References Abutaleb S., 1989. Automatic thresholding of gray-level picture using two dimensional entropy. Comput. Vision Graphics Image Process 47, 22–32. Ardizzone E., Dindo H., Mazzola G., 2009. Recent Advances in Signal Processing, in Ch. 9: Content-Based Image Retrieval as Validation for Defect Detection in Old Photos. Intech, 147-158. Chang R. C., Sie Y. L., Chou S. M., Shih T. K., 2005. Photo Defect Detection for Image Inpainting. Proceedings of the Seventh IEEE International Symposium on Multimedia (ISM’05), pp.1-5. Hou Z., Hub Q., Nowinskib W.L., 2006. On minimum variance thresholding. Pattern Recognition Letters 27(14), 1732–1743. Hui Fuang Ng, 2006. Automatic thresholding for defect detection. Pattern Recognition letters, Science direct 27(14), 1644–1649.
Fig. 12. Multi colored defected photo. a. Day images, b. Night images, c. Result image d. Histograms.
Hui Fuang Ng, Rahman Tunku Abdul, Jargalsaikhan, D., Tsai Hao Chuan, Lin Chih-Yang, 2013. An improved method for image thresholding based on the Valley emphasis method. IEEE’s Signal and Information Processing Association Annual Summit and Conference (APSIPA), Asia Pacific, Kaohsiung, INSPEC Accession Number: 14002970, 1– 4.
ACCEPTED MANUSCRIPT
8
Jiu Lun Fan and Bo Leia, 2012. A modified valley emphasis method for automatic thresholding. Pattern Recognition Letters, Science direct 33 (6), 703–708.
Portes de Albuquerque M., Esquef I.A., Gesualdi Mello A.R., Portes de Albuquerque M., 2004. Image thresholding using Tsallis entropy. Pattern Recognition Letters, 251059–1065.
Kapur J.N., Sahoo P.K., Wong A., 1985. A new method for graylevel picture thresholding using the entropy of the histogram. Computer Vision Graphics Image Process 29, 273–285.
Saha P.K., Udupa J.K., 2001. Optimum image thresholding via class uncertainty and region homogeneity. IEEE Trans. Pattern Anal. Mach. Intell. 23 (7), 689–706.
Kittler J., Illingworth J., 1986. Minimum error thresholding. Pattern Recognition 19 (1), 41–47.
Sahoo P.K., Arora G., 2004. A thresholding method based on two-dimensional Renyi’s entropy, Pattern Recognition Letters, 37, 1149–1161.
Maddalena L., Petrosino A., Laccetti G., 2009. A fusion-based approach to digital movie restoration. Pattern Recognition 42 (7), 1485-1495. Morii Fujiki, An image thresholding method using a minimum weighted squared-distortion criterion, 1995. Pattern Recognition 28(7), 1063–1071. Newman, T.S., 1995. A survey of automated visual inspection. Computer Vision and Image Understanding 61 (2), 231–262.
Wanga Shitong, Chungb Fu-lai, Xionga Fusong, 2008. A novel image thresholding method based on Parzen window estimate. Pattern Recognition 41, 117 – 129. Xiangyang Xu, Shengzhou Xu, Lianghai Jin, Enmin Song, 2011. Characteristic analysis of Otsu threshold and its applications. Pattern Recognition Letters 32(7), 956–961. Yasnoff, W.A., Mui, J.K., Bacus, J.W., 1977. Error measures for scene segmentation. J. of Pattern Recognition 9, 217–231
AC
CE
PT
ED
M
AN US
Otsu N., 1979. A threshold selection method from gray-level histograms. IEEE Transactions on Systems Man Cybernet SMC9 (1), 62–66.
Sezgin M., Sankur B., 2004. Survey over image thresholding techniques and quantitative performance evaluation. J. of Electronic Imaging 13 (1), 146–156.
CR IP T
Kokaram, AC., 2004. On missing data treatment for degraded video and film archives: a survey and a new Bayesian approach. IEEE Transactions on Image Processing 13 (3), 397-415.
An imaging approach for the automatic thresholding of photo defects Neelam Bhardwaj , Prof. Suneeta Agarwal , Vikash Bhardwaj PII: DOI: Reference:
S0167-8655(15)00112-9 10.1016/j.patrec.2015.03.019 PATREC 6202
To appear in:
Pattern Recognition Letters
Received date: Accepted date:
31 August 2014 23 March 2015
Please cite this article as: Neelam Bhardwaj , Prof. Suneeta Agarwal , Vikash Bhardwaj , An imaging approach for the automatic thresholding of photo defects, Pattern Recognition Letters (2015), doi: 10.1016/j.patrec.2015.03.019
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
1
Highlights
Evaluation on limitations of valley emphasis method for thresholding of photo defects. Proposed technique of photo defect detection involves imaging of defected photos by sensing the light after passing through them instead of sensing the reflection from their surfaces and then applying valley emphasis method to threshold defect objects.
The proposed approach is able to locate the defects with very small misclassification error without requirement of any defect object features. It overcomes the limitation of thresholding techniques to distinguish the defect objects in presence of similar photo objects and in cases of defects having very low contrast with their back ground (unimodal histogram). The obtained threshold values with proposed approach for all cases of additive defects are less than 50. It reduces the time complexity of defect detection by least 1/5 times as the dynamic range of intensity levels for defected pixels in histogram confine to 0-50 instead of 0-255. Presently the proposed approach is applicable only for physically available photos having light transitivity, but imaging with IR and x-ray may also be explored to overcome the current limitation.
AC
CE
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
An Imaging Approach for the Automatic Thresholding of Photo Defects Neelam Bhardwaj1a, Prof. Suneeta Agarwal 2b, Vikash Bhardwaj 3c a
PhD scholar, Computer Science, MNNIT, Allahabad, UP, INDIA Prof. (Dr.) Suneeta Agarwal, Computer Science, MNNIT, Allahabad, UP, INDIA c Scientist, ADRDE, Agra, UP, INDIA b
CR IP T
ABSTRACT
AN US
Automatic thresholding of photo defects means to accurately locate defect objects. The available approaches for automatic thresholding determine the optimal threshold values and segment the image into objects based on their gray level distribution. For defect object identification in images with multimodal distributions, these techniques also require knowledge of the defect object features, such as shape and size, which limits the applicability of these techniques because the defect object features may vary widely. Additionally, these methods result in extensive misclassification errors in the presence of photo objects similar to defect objects and unimodal distribution. We evaluated the limitations of the valley emphasis method and proposed a new approach that involves the imaging of a defected photo by sensing the light after it passes through photo and then applying the valley emphasis method for thresholding to identify defect objects. The obtained results are better even with the above discussed constraints of the available automatic thresholding approaches. Although the proposed technique is applicable only for physically available objects, it may contribute significantly towards the accuracy of machine vision based applications.
AC
CE
PT
ED
M
Key words: Thresholding, photo defect detection, imaging, valley emphasis method.
Tel.: +91-09411464637; e-mail: [email protected]
ACCEPTED MANUSCRIPT
1. Introduction
2
In the spatial gray level distribution domain, Wanga et al. (2008) attempted to explore a novel global image thresholding method based on the Parzen window estimate of an unknown gray value probability density function rather than the gray level histograms of an image. The Parzen window effectively integrates image histogram information with explicit spatial information about pixels of different gray levels. The MHUE thresholding method ((Saha and Udupa, 2001) utilizes the spatial information of an image. In this method, the spatial information is implicitly represented by region homogeneity. L. Maddalenaa et al. (2009) used a fusion-based detection approach for scratches in corrupted image sequences. They fused algorithms utilizing Radon projection and then detected local maxima using Top Hat transformation and developed a scratch mask by assigning value 1 to all the pixels of columns that contain more than 70% pixels with value 1. Anil C. Kokaram (2004) presented his work on the restoration of defected motion pictures, where he also discussed the Spike Detection Index (SDI), Rank Order Detector (ROD), and image histogram based approaches and morphological approaches for defect detection.
AN US
The basic idea of thresholding is to find an optimal gray-level by analyzing the gray-level distribution and extracting the objects of interest from an image. There are two types of automatic thresholding techniques, global thresholding and local thresholding. In global thresholding, a single threshold value is taken from the histogram of the entire image, and in local thresholding, the local gray-level distributions are explored to determine multiple threshold values. For defect detection applications, global thresholding is easier to implement, but its applicability is limited in photo images with distribution close to unimodal, whereas local thresholding methods work for both nearly unimodal and multimodal distributions, but they are complex.
with a smaller variance. To address this problem, they proposed an improved Otsu algorithm that constrains the search range of the gray levels. The minimum error thresholding (MET) method also assumes that the image can be characterized by a mixture distribution of foreground and background pixels (Kittler and Illingworth, 1986). The entropy-based techniques exploit the entropy of the distribution of the gray levels and interpret the maximization of the entropy of the thresholded image as indicative of maximum information transfer. The maximum cross entropy method (Kapur et al., 1985 and Abutaleb, 1989) considers the foreground and background of an image as two different information sources, so that when the sum of these two class entropies reaches its maximum, the image is said to be optimally thresholded. The Tsallis entropy-based method (Portes de Albuquerque et al., 2004), Renyi’s entropy-based method (Sahoo and Arora, 2004), and supervised Lloyd algorithm (Morii, 1995) are also typical examples of this class.
CR IP T
Photo defect detection is the foremost essential requirement for the development of accurate images. Generally, photos are damaged by the occurrence of defects either by the addition of layers of substance (additive defects) or by the removal of layers of substance from their surfaces. The defect standard taxonomy is available (Ardizzone et al., 2009). Defects alter the intensity values of pixels at their locations and can be identified with the help of optimal intensity thresholding. Automatic thresholding of defects reduces the span of the histogram for analysis to locate defect objects. This enhances the effectiveness of vision-based industrial applications and may facilitate real-time corrective action in the manufacturing phase itself.
M
Because of its scope, which includes other areas of image processing and machine vision based applications, much work on automatic thresholding has already been conducted. Recently, Sezgin and Sankur (2004) conducted an exhaustive survey of non-parametric and statistical approaches for image thresholding, where they categorized the approaches according to the information used, such as the histogram shape, measurement space clustering, entropy, object attributes, spatial correlation, and local gray-level surfaces and compared their performance.
AC
CE
PT
ED
Among clustering-based methods, where the gray level samples are clustered in two parts as the background and foreground, the Otsu method (Otsu, 1979) is the most popular method. This method selects threshold values that maximize the between class variances of the histogram. This technique works well but finds limitations in the case of unimodal or close to unimodal gray-level histograms. Many modifications have been added to the Otsu method to overcome its limitations. Hui Fuang Ng (2006) revised the Otsu method as the valley-emphasis method, in which the valley point information is considered in the objective function. In this method, the optimal threshold is decided by maximizing the between class variance and minimizing the within class variance. In this method, threshold values are located as close as possible to the valley points in the histogram. The modified valley emphasis method (Jiu Lun Fan and Bo Lei, 2012) covers the limitation for the case where the variance of the object is different from that of the background. They considered not only the valley-point information but also the neighborhood gray information around the valley-point and selected a threshold value that has small probabilities in its neighborhood area and also maximizes the between class variance in the gray-level histogram. Hui Fuang Ng et al. (2013) further improved the valley-emphasis method by introducing a Gaussian weighting scheme that efficiently uses the neighborhood information for inclusion in the objective function to enhance the effect of the weight factor. Hou et al. (2006) proposed minimum class variance thresholding (MCVT) and compared it with the Otsu method. They found that in the Otsu method, the resulting threshold is biased towards the component with a larger class variance or larger class probability. Xiangyang Xu et al. (2011) also described that when the within class variances of two classes are different, the Otsu threshold is biased toward the class with a larger variance. As a result, partial pixels belonging to this class will be misclassified into the other class
Chang et al. (2005) worked in a different way to threshold defects due to ink spray and scratches by thick colored pens. They used the low intensity variance property of ink spreading at defect locations and designed a thresholding filter to determine low varying and smooth portions of the histogram. Then, they tracked the decreasing rate of the area of thresholded objects. If the decreasing rate is high, then an object is not likely to be a defect object, and if the reduction in an object's size is less than half of the intensity value reduction of one, it is claimed as a defect object. All the above discussed methods of automatic thresholding only produce the optimal segmentation of a defected input image into its constituent objects but cannot claim that a particular object is a defect object. Still, in the case of bimodal distributions, the defect objects can be filtered based on size. However, for defect detection in the case of a multimodal distribution, the entire image is scanned for comparison with available defect object properties for the best match, which makes these methods more complex and time consuming. The requirement of defect object features related to their shape and size limit the applicability of these techniques, as defect features are random in nature. These methods also generate high misclassification error in the case of similarity of defect objects to their background and undamaged image objects. Automatic thresholding has been widely used in the industry for automated visual inspection of defects (Newman, 1995). If a technique with less time complexity and independent to defect properties is available, it will enhance the effectiveness of visionbased applications and make them able to take real time
ACCEPTED MANUSCRIPT
We have proposed a new technique of photo defect detection that images defected photos by sensing the light that passes through them instead of by sensing the reflection from their surfaces to applying the valley emphasis method to threshold defect objects. Imaging of a defected photo by the proposed technique enhances the contrast of defect objects with the image objects. In fact, a unimodal distribution becomes bimodal or multimodal, which makes them appropriate inputs to undergo automatic thresholding for accurate detection. We have experimented with approximately 20 input photos defected with various types of additive defects. We have also evaluated the limitations of the valley emphasis method and compared the result with the proposed method by taking two cases of suitably defected input photos. The results obtained are even better with above discussed constraints of currently available approaches. Due to the enhanced contrast of defect objects with image objects, this approach is able to distinguish the defect object from similar image objects. Although the proposed technique is only applicable to physically available photos, it may also contribute significantly towards the accuracy of machine vision based applications.
Fig. 2. Experimental setup
AN US
In this paper we have used the term "day image" for an image acquired by scanning the defected photo image by a conventional method i.e., by sensing the light reflection, "night image" for an image acquired by imaging the defected photo with the proposed method i.e., by sensing the light passed through the defected image, and "additive defect" for a defect caused by the addition of defect material.
For experiments, we prepared a setup as shown in figure 2 with the help of a wooden box, a white 5 W CFL as a light source, a 4 mm thick transparent glass as a base for holding the subject photos and 8 megapixel Sony cameras for image acquisition. The vertical distance between the light source and transparent glass/photo is kept fixed. For better results, the imaging is performed in a dark room to minimize the effect of light reflection.
CR IP T
corrective action at the manufacturing phase to reduce waste during inspection.
3
M
In this paper, after an introduction about the problem and related work, Section II introduces the proposed imaging technique, Section III provides details about obtaining the threshold value, Section IV describes the details of the experiments and the contribution made, and the report is finally concluded in Section V.
ED
2. Proposed approach
Having a defected photo physically available as input, the goal is to locate the defect objects and mark them with an appropriate color to highlight their presence. The overall procedure is illustrated in figure 3. During night image acquisition using the experimental set up, camera flash and room lights are kept OFF, while the light source of the imaging setup is kept ON. For day image acquisition, the light source was kept OFF, while the camera flash and room lights were kept ON. The camera position and settings are fixed to obtain both types of the images of the same size and with one to one mapping. After the acquisition of the images, the gray level distributions of the night images are analyzed for the optimal threshold value to locate the defected pixel, and then these detected pixels are highlighted in the corresponding day image for better appearance. The image processing is carried out in the luminous region using MATLAB.
AC
CE
PT
Generally, day images are used for thresholding in defect detection. However, in the proposed approach, the night images are explored primarily and used as inputs for the valley emphasis method for deciding the optimal threshold value. Due to the presence of additive defects, the image thickness at defect locations increases compared to the remaining undamaged regions. This keen observation led us to develop an entirely different approach than that found in the literature, where we have developed a new imaging setup for sensing the light passed through the defected photos instead of the usual way of scanning by sensing the reflected light from their surface. The proposed approach involves night image and day image acquisition followed by analysis and processing. The details about the proposed approach are given in the following subsections.
2.2. Defect detection procedure
Fig. 3. Defect detection procedure
2.1. Proposed Imaging and Experimental setup
3. Automatic thresholding of defects
As the proposed imaging approach is based on sensing the light after passing through the input photos, light is projected from one side and sensed after passing through the subject photos by an image acquisition device placed on the opposite side, as shown in figure 1.
3.1. Theoretical estimates of defect locations
Fig. 1. Proposed imaging approach
The spatial intensity distribution of the light passed through the photos varies directly according to their spatial thickness variation. In the case of additive defects, the surface thickness at the defected locations of the photos becomes more, and in the case of scratches, the surface thickness at the defected area becomes less. In both cases, the thickness of the photos at the defected locations becomes different compared to the remaining undamaged portion of the photos. Due to this effect of a defect on photo thickness, the defect locations boost their contrast in the luminous intensity domain of the night images.
4
ACCEPTED MANUSCRIPT
For example, if the photo thickness at the defected area becomes double or half compared to the undamaged locations, then according to the Beer – Lambert law, the intensity of the night image at the defected locations will be less by a factor of ‘102α’ in the case of additive defects and more by the same factor in the case of scratch defects, where α = absorption coefficient of the material. The night image, defected at location (i, j) in the luminous domain, may be described using equation (1) and equation (2).
( )
( )
( )
( ) ( )
{
where G(i, j) is the night image intensity in the presence of the defect, I(i, j) is the night image intensity at the defected location in the absence of the defect, D(i, j) is the effect of the defect at location (i, j), ‘K’ is the sensitivity of the camera focal plane array and I0 is the intensity of the incident light.
Fig. 4. Black and white undamaged photo. (a) Day image, (b) Night image (c) Relative histograms
In the second case, defect objects are removed manually from the night images. Figure 5 shows that in the first case, the number of pixels on the lower side is higher than in the second case. The sharp rise at the upper side in the histogram in second case is because of the intentional replacement of defect objects with white color. Thus, it is validated that in the night image histograms, only the additive defect objects are confined to the lower most side within the intensity limit of less than 50.
M
AN US
Due to the defects, factor ‘D’ affects the intensity at the defect locations significantly and shifts the defected objects either at the lower side in the case of additive defects or at the upper side in case of scratches in the luminous intensity histogram of the night image. More values of the defect material thickness and its absorption coefficient would increase the effectiveness of factor ‘D’ and produce better results. The undamaged photo objects of dark colors and light colors also shift their locations toward the interior from both sides in the night image histogram compared to their locations in the corresponding day image histograms. This is shown in figure 4. Thus, only the defect objects of the photos are confined at the lower most and uppermost regions in the luminous histogram of the night image. Hence, it is estimated that in the proposed imaging, the additive defect regions are confined within the lower most lobe of the multimodal distribution.
CR IP T
( )
3.2. Validation of theoretical estimate
ED
For the validation of above estimates, we have taken various input photos and compared the night images with the corresponding day images. The outcomes are discussed in the following subsection.
PT
3.2.1. Contraction of night image histogram for undamaged photos
AC
CE
In the previous section, it is theoretically estimated that the undamaged photo objects ranging from dark to light colors also shift their locations toward the interior from both sides of the night image histogram compared to their locations in the corresponding day image histograms. To quantify this, we compared the histograms of the day and night image of a black and white undamaged input photo. A black and white colored photo is preferred because these color pixels are present at the extreme ends of the day image histogram of any multi colored photo. Figure 4 shows that in the night image histogram of same photo, the lowest intensity pixels of the corresponding day image shift by a factor of approximately 30 away from origin, and the highest intensity pixels shift towards the origin by a factor of approximately 10. Thus, the undamaged photo objects of dark colors and light colors also shift their locations toward interior from both sides in night image histogram compared to their locations in the corresponding day image histograms. 3.2.2. Effect of defect objects on the night image histogram To validate that only additive defect objects of photos are confined at the lowermost side of the luminous histogram of night image, we compared two cases of night image histograms of photos with additive defect objects. One case includes the defect objects, and other case does not include the defect objects.
3.3. Threshold value
With the above conclusion that only additive defect objects are present on the lower side of the night image histogram, we take the difference between the night and day image histograms as an input to the valley emphasis method and also imposed the intensity limit of a maximum of 50. The difference between the histograms is taken because it has clear valley points, and only the additive defect objects are emphasized in the lowermost lobe. The basis of the valley-emphasis method is to select a threshold value that has a small probability of occurrence, i.e., a valley in the gray-level histogram, and also maximize the between group variance, as in the Otsu method. The detail of the formulation of the proposed method is described below. The gray level histograms of the night image and day images can be represented by the intensity functions f n(i) and fd(i), where i ranges from 0 to L 1, and L is the number of distinct graylevels. Then, the difference function f(i) is taken as the input to the Valley method and defined as ()
* ()
( )+
( )
If the number of pixels at the ith gray level in f(i) is represented by ni and n is the total number of pixels in ( ), then the probability of the occurrence of the ith gray-level is defined as⁄
( )
The average gray-level of ( ) is defined as: ∑
( )
5
ACCEPTED MANUSCRIPT
Fig. 5. Night image histograms with the presence and absence of additive defects. a, b & c correspond to the test images of figs. 10, 11 & 12, respectively
|
The single thresholding means that the image pixels are grouped into two classes separated by a threshold value,‘t’.
()
()
∑
( )
∑
and the mean gray-level values of these classes can be computed as ∑
⁄
()
()
∑
⁄
() ( )
Using discriminate analysis, Otsu showed that the optimal threshold t can be determined by maximizing the between-class variance, i.e., ()
{
()
()
( )}
( )
()
()
()
( ))} ( )
ED
)(
M
Later, Valley modified equation (8) for the optimal threshold ) as given in value calculation by adding a weight factor ( equation (9): {(
|
|
| |
Where FO denotes the foreground area pixels or the manually thresholded defect object area (pixels) using Adobe Photoshop software, and BO denotes the background area pixels or the rest of the image area other than the defect pixels. FT denotes the detected pixels as defected pixels, and BT denotes the unmarked defected pixels using the proposed approach. Basically, this misclassification error reflects the percentage of background pixels wrongly assigned to the foreground, and conversely, foreground pixels wrongly assigned to the background. It may range from zero for no error, to one for completely incorrect detection. For the evaluation of each case of defect detection,
first we obtained the optimal threshold value using the proposed approach and then applied this threshold value to the corresponding night images containing the defect objects and again applying to the same night image after manually removing defect objects from it using software such as Adobe Photoshop to identify false detection and false rejection. The experimental results are discussed as three cases of input images to analyze the efficacy of proposed approach compared to the Valley emphasis method.
AN US
()
|
CR IP T
Then, the probabilities of these two classes are:
|
|
The proposed approach is able to identify and locate defect objects having very little contrast with their background. The day images of these types of defected photos have a unimodal gray level distribution, whereas their corresponding night images have at least bimodal gray level distribution. The outcomes of such examples are shown in figures 6 and 7. The misclassification errors are calculated after applying the valley emphasis method on day image histograms and on the difference between night and day image histograms, as given in table 1. It is observed that the proposed approach, in this case, is able to locate defected pixels
AC
CE
PT
The key of the valley-emphasis formulation is the weight ). is the probability of the occurrence of the tth grayfactor( level. The smaller the value is, the larger the weight value is. This weight ensures that the resulting threshold always resides at the valley of the gray-level distribution. The proposed approach limits the value of t in equation (9) to be less than 50 and emphasizes only the first effective valley point as a threshold value. Thus, the number of iterations to find the threshold value is at least 5 times less compared to the existing approach of the Valley method. The modified expression is given in equation (10):
4.1. Case1: Defected photo with unimodal gray level distribution
{(
)(
()
()
()
( ))} (
)
The optimal threshold values obtained with the proposed approach and the corresponding misclassification errors for the defected input photos are given in tables 1, 2 and 3 in the results section of this paper. 4. Results and Contributions Approximately 20 defected photos on a white sheet as a base and defected with various additive defects were collected as input. Encouraging results for defect detection were obtained. We evaluated the performance of the proposed method by calculating the misclassification errors using the definition (Yasnoff et al., 1977) as-
Fig. 6. Photo where defect objects have low contrast with background. a. Day image, b. Night image, c & d. Histograms with threshold marked, e & f. Result.
ACCEPTED MANUSCRIPT
6
Table 1. Results for applying valley emphasis method on day and night images of defected photo with unimodal histogram
Photo
Size
Size of defects
Figure 6 Figure 7
233562 258640
20551 55042
Valley Threshold value Day Night image image 31 106 17 103
Detected defect pixels False detection Day image 231773 258351
Night image 20699 55138
Night image 150 134
4.3. Case 3: Defected photos with Multimodal gray level distribution We have tested the proposed scheme on more than 20 multi colored and multi object photos with additive defects. Some of the results are shown in figures 10, 11 and 12. The misclassification errors in these cases are given in table 3. The proposed approach is able to threshold the defect objects from the remaining undamaged portion and obtain the threshold value of less than 50. If we apply the Valley method on day images of
CR IP T
with very small misclassification error, whereas the Valley emphasis method, if applied on a day image gives very high misclassification error, and in fact categorizes defect objects and background as a single object, as shown in figure 6.e and 7.e marked with a white color.
Day image 210276 202516
Misclassification error Day Night image image 0.8963 0.00065 0.7799 0.00066
AN US
the subjected photos with multimodal distribution, only the segmented components will be obtained, but it is not possible to declare any of the particular classes of objects as defects without prior knowledge of the defect.
Fig. 7. Photo where defect objects have low contrast with background. a. Day image, b. Night image, c & d. Histograms with threshold marked, e & f.
M
4.2. Case2: Defected photos with multimodal gray level distribution where defect objects have similarity with undamaged photo objects
Fig. 9. Photo with defect object similar to images objects in terms of color. a. Day image, b. Night image, c & d. Histograms with threshold marked, e & f. Result images.
AC
CE
PT
ED
We have generated the test images as shown in figure 8.a and 9.a. In the first test case, the blue colored circular object at the center is intentionally placed as a defect object, which is similar to the objects at the corners of the original photo, and in the second case, a multicolored photo is defected with a black color that is almost similar to the background at its location. The results of this case are given in table 2. Although the day images of these types of defected photos have a multimodal gray level distribution, the application of the valley emphasis method on the day images of these photos categorize the defect objects in same class as similar image objects, whereas the proposed approach in such cases is able to locate and distinguish the defected pixels from similar undamaged image pixels with very small misclassification error. The outcomes of these examples are shown in the result images of figures 8 and 9.
Fig. 8. Photo having defect object similar to images objects. a. Day image, b. Night image, c & d. histograms with threshold marked, e & f. result images.
Fig. 10 .Gray colored photo defected with black and white color. a. Day images, b. Night images, c. Result image d. Histograms.
ACCEPTED MANUSCRIPT
7
Table 2. Results for applying valley emphasis method on day and night images of defected photo, where defect object has similarity to undamaged photo objects Valley Threshold Misclassification Detected defect pixels False detection value error Size of Photo Size defects Day Night Day Night Day Night Day Night image image image image image image image image Figure 8 3273450 75182 89 23 254030 65313 186468 0 0.0593 0.0030 Figure 9 303094 7267 101 12 211952 8019 203324 3026 0.6663 0.0175 Table 3. Results for applying valley emphasis method on night images of defected photo with multimodal histogram Size of Valley Threshold Detected defect Misclassification Photo Size False detection defects value pixels error Figure 10 782738 64571 20 64841 281 0.000373 Figure 11 1593584 99492 35 99403 45 0.000143 Figure 12 1857136 130802 36 136464 5666 0.0031
4.4. Contributions
AN US
CR IP T
The proposed approach is able to locate the defects with very small misclassification error without the requirement of any defect object features. It overcomes the limitation to distinguish the defect objects in the presence of similar photo objects and in cases of defects having very low contrast with their back ground. The obtained threshold values for all cases are less than 50, which reduces the time consumption of defect detection by least 1/5 times, as the dynamic range of intensity levels for the defected pixels in the histogram are confined to 0-50 instead of 0255. Thus, the proposed approach will enhance the performance of machine vision based applications.
ED
M
5. Conclusion and future scope
AC
CE
PT
Fig. 11. Multi colored defected photo. a. Day images, b. Night images, c. Result image d. Histograms.
The results obtained and presented above are encouraging. The performance of the proposed approach is better even with the current limitations of thresholding approaches. The approach proposed in this paper is also able to distinguish the defect objects from similar image objects. The proposed defect detection consumes less time and prompts its applicability in vision based industrial applications towards an automated correction mechanism during the production phase itself to save waste during the later inspection stage. The work to apply the proposed approach towards scratch detection is also in progress. In the future, imaging with light in the IR and x-ray regions may also be explored to overcome the current limitation of the proposed method i.e., photos with non-light transmitting bases. References Abutaleb S., 1989. Automatic thresholding of gray-level picture using two dimensional entropy. Comput. Vision Graphics Image Process 47, 22–32. Ardizzone E., Dindo H., Mazzola G., 2009. Recent Advances in Signal Processing, in Ch. 9: Content-Based Image Retrieval as Validation for Defect Detection in Old Photos. Intech, 147-158. Chang R. C., Sie Y. L., Chou S. M., Shih T. K., 2005. Photo Defect Detection for Image Inpainting. Proceedings of the Seventh IEEE International Symposium on Multimedia (ISM’05), pp.1-5. Hou Z., Hub Q., Nowinskib W.L., 2006. On minimum variance thresholding. Pattern Recognition Letters 27(14), 1732–1743. Hui Fuang Ng, 2006. Automatic thresholding for defect detection. Pattern Recognition letters, Science direct 27(14), 1644–1649.
Fig. 12. Multi colored defected photo. a. Day images, b. Night images, c. Result image d. Histograms.
Hui Fuang Ng, Rahman Tunku Abdul, Jargalsaikhan, D., Tsai Hao Chuan, Lin Chih-Yang, 2013. An improved method for image thresholding based on the Valley emphasis method. IEEE’s Signal and Information Processing Association Annual Summit and Conference (APSIPA), Asia Pacific, Kaohsiung, INSPEC Accession Number: 14002970, 1– 4.
ACCEPTED MANUSCRIPT
8
Jiu Lun Fan and Bo Leia, 2012. A modified valley emphasis method for automatic thresholding. Pattern Recognition Letters, Science direct 33 (6), 703–708.
Portes de Albuquerque M., Esquef I.A., Gesualdi Mello A.R., Portes de Albuquerque M., 2004. Image thresholding using Tsallis entropy. Pattern Recognition Letters, 251059–1065.
Kapur J.N., Sahoo P.K., Wong A., 1985. A new method for graylevel picture thresholding using the entropy of the histogram. Computer Vision Graphics Image Process 29, 273–285.
Saha P.K., Udupa J.K., 2001. Optimum image thresholding via class uncertainty and region homogeneity. IEEE Trans. Pattern Anal. Mach. Intell. 23 (7), 689–706.
Kittler J., Illingworth J., 1986. Minimum error thresholding. Pattern Recognition 19 (1), 41–47.
Sahoo P.K., Arora G., 2004. A thresholding method based on two-dimensional Renyi’s entropy, Pattern Recognition Letters, 37, 1149–1161.
Maddalena L., Petrosino A., Laccetti G., 2009. A fusion-based approach to digital movie restoration. Pattern Recognition 42 (7), 1485-1495. Morii Fujiki, An image thresholding method using a minimum weighted squared-distortion criterion, 1995. Pattern Recognition 28(7), 1063–1071. Newman, T.S., 1995. A survey of automated visual inspection. Computer Vision and Image Understanding 61 (2), 231–262.
Wanga Shitong, Chungb Fu-lai, Xionga Fusong, 2008. A novel image thresholding method based on Parzen window estimate. Pattern Recognition 41, 117 – 129. Xiangyang Xu, Shengzhou Xu, Lianghai Jin, Enmin Song, 2011. Characteristic analysis of Otsu threshold and its applications. Pattern Recognition Letters 32(7), 956–961. Yasnoff, W.A., Mui, J.K., Bacus, J.W., 1977. Error measures for scene segmentation. J. of Pattern Recognition 9, 217–231
AC
CE
PT
ED
M
AN US
Otsu N., 1979. A threshold selection method from gray-level histograms. IEEE Transactions on Systems Man Cybernet SMC9 (1), 62–66.
Sezgin M., Sankur B., 2004. Survey over image thresholding techniques and quantitative performance evaluation. J. of Electronic Imaging 13 (1), 146–156.
CR IP T
Kokaram, AC., 2004. On missing data treatment for degraded video and film archives: a survey and a new Bayesian approach. IEEE Transactions on Image Processing 13 (3), 397-415.