Adaptive Threshold Estimation for The Inspection of Hot Steel

Copyright © IFAC Identifi cation and Syste m Pa ramete r Estima tion 1982. W as hington D.c.. USA 1982

ADAPTIVE THRESHOLD ESTIMATION FOR THE INSPECTION OF HOT STEEL R. A. Fundakowski, T. S. Levitt, B. R. Suresh and T. M. Wittenburg Honeywell Inc., Syst ems & Research CenteT, 2600 Ridgway PaThway, P. O. Box 312, Minn eapolis, Minn esota 55440, USA

Abstract. This paper outlines a practical method of automating the threshold selection for segmentation in a real-time image processing system. This systeM has been developed to perform on-line surface inspection of hot steel slabs, thereby reducing substantially the energy consumption of the steel industry (largest single user in the U.S.). The threshold selection scheme has been designed to provide suitable clutter rejection while regulating the system processing backlog. The discussion proceeds along the line of adaptive threshold design rationale, system architecture constraints and the ensuing implementation. Performance of the adaptive threshold is compared to that of fixed threshold schemes. Keywords. Adaptive systems, control engineering computer applications, industrial control, manufacturing processes, pattern recognition, signal processing, steel industry, steel manufacture, digital image processing. The development of a real-time inspection system for this application poses a significant technical challenge. The real-time throughput rate of 546 Kpixels/sec. (determined by resolution requirements and slab velocity) , although an order of magnitude below standard video rates, is vastly beyond the capabilities of a general purpose minicomputer. Secondly, the flexibility desired in algorithm development makes the design and build of high speed special purpose hardware out of the question.

INTRODUCTION Automatic inspection systems promise a large potential impact on many industries. In particular, the hot inspection of steel slabs would eliminate the need for cold manual inspection and subsequent reheating prior to reworking, thereby reducing substantially the energy consumption of the steel industry. Currently, in most steel mills, the hot slabs produced must first be cooled in order to manually inspect the slab surface for imperfections such as cracks and tears. Once the slab has been determined to be free of significant imperfections it must then be reheated for further processing. An on-line inspection system capable of performing hot inspection of the steel slabs would resul t in enormous cost savings by virtue of the energy saved in avoiding intermediate cooling and reheating (lM BTU/ton) and more importantly would open the door to the possibility of automated in-line conditioning as a response to the inspection system diagnosis.

For this reason, a unique instrumentation system consisting solely of off-the-shelf components has been designed (Suresh and colleagues, 1982). The architecture of the proposed real-time image processing system, incorporating an array processor as a high speed front end and a general purpose minicomputer, is shown in Fig. 1. The MAP-300 processor has two primary functions. First it performs object segmentation by edge enhancement and edge value

2 JJ

R. A. Fundakowski et al.

212

thresholding, and secondly, it performs object labeling and feature extraction by sequential scan line processing. The host, a Honeywell Level 6 minicomputer, then performs classification and generates a ~eport d~tailing the detected slab lmperfectlons and the determined disposi tion (Suresh, Fundakowski and Levitt,1981). The purpose of object segmentation is to identify regions corresponding to possible surface anomalies. In this application, most surface imperfections are manifested as linear objects embedded in a high clutter background. In most practical applications, clutter is the single largest contributor to the exacerbation of the real-time performance problem. Unfortunately, clutter rejection techniques usually come at the expense of obscuring the primary objects of interest. A trade-off is thus involved between clutter rejection and the recovery of the linear object. The remainder of the paper deals with the method by which the edge value thresholding scheme critical to object segmentation has been designed and implemented. DESIGN RATIONALE On each scan line, which consists of 2048 pixels, regions of interest (i.e., segments of possible imperfections) are extracted. These intervals of interest on each scan line are determined by comparing the edge value at each pixel on the scanline with a sui table threshold. If the edge value exceeds the threshold, then the pixel is I?resumed to contribute to an object lnterval for that line. Information of this form from the entire scan line is. distilled for the subsequent processlng stages. The selection of a proper threshold is crucial in order to cut down on clutter while retaining the pr imary objects of interest. In examining the slab imagery, it was observed that the slab surface could be divided into 3 homogeneous zones based on the imagery statistics. These three distinct zones come about because of the peculiar geometry of the rollers in the steel mill caster. However, within a given zone, the image statistics were observed to be fairly stationary. For this reason a different thresholding scheme wa~ employed in each zone. A fixed

threshold was applied in the first two zones, with a different threshold being used for each zone. However, an adaptive threshold was found to be necessary for the third zone in order to provide an acceptable level of performance. The threshold adjustment algorithm needs to be autonomous while not jeopardizing the real-time performance of the system with its sophistication. The tight time constraints on the front-end algorithms (edge value calculation, thresholding and interval generation) make edge value t .hreshold adjustment on the individual pixel basis impractical. The development of an appropriate adaptive threshold is presented here in two stages. First a recursive edge value threshold is presented that reflects both the need for clutter rejection as well as the array processor real-time throughput constraints. Modifications of the scheme for the purposes of controlling the processing backlog are then outlined. RECURSIVE THRESHOLD CALCULATION Across the scan line of 2048 pixels the edge values E' (j=1,2, .•• ,2048) of individual pixels are summed. A smoothed estimate of the average edge value for the i+lth scan line, SA{i+l), is then generated from an IIR filter according to SA(i+l)

+

(1- Cl)

SA(i)

(1)

[Ej

(Cl/N)

j

where

0 <

Cl

< < 1.

This smoothing desensitizes the edge value sum to changes in edge values on isolated scan lines. An IIR filter was chosen because it requires only one result SA(i) to be saved from one scan line to the next. The edge value threshold the i-th line scan determined by Te(i)

K

K = {KH KL

if

if

(Te) for is then

SA(i) .

(2)

SA < SAcrit SA > SAcri t.

The application of the multiplier K as shown above acts as an area imperfection detection scheme. That is, when the smoothed average SA

Adaptive Threshold Estimation

exceeds a prescribed value SAcrit, then the edge values observed are not those of isolated linear imperfections but rather attributed to large textured area imperfections. In this case a multiplier of lower value (KL) is applied in place of the higher valued multiplier (KH). A lower bound is subsequently imposed on the threshold Te(i) so that the threshold cannot be reduced to a point where only noise is detected. The preceding scheme for the adjustment of the edge value threshold requires only one addition per pixel and can, in fact, be completely hidden behind other operations in the object segmentation process. QUEUE BASED THRESHOLD MODIFICATION Another desirable feature of the threshold adjustment would be the regulation of the processing backlog at subsequent stages of the system. Of particular importance is the processing backlog at the object labeling and feature extraction stage where edges extracted during segmentation are tracked and features are extracted. The vector of edge points that are to be tracked by the edge tracking and labeling routine are stored in a circular buffer. This buffering of segmentor output is required because of the asynchronous processing rates of the segmentation and the object labeling and feature extraction stages ongoing in two parallel processors within the array processor. When this fixed size buffer is full (indicating that the labeler/feature extraction has fallen behind an amount equal to the buffer size) either stage will have to "skip" scan lines to rectify the situation. Such a procedure is too radical a means for the queue adjustment, in that entire scan lines of information will be sacrificed. For this reason, a less drastic control of the queue is preferable. Let us assume that the threshold given by equation (2) is desirabl~ solely from the standpoint of detection and clutter rejection, and outline a suitable strategy to modify this threshold for the sake of controlling the depth of the

processing queue (vectors of points that have not yet processed) .

213

edge been

Ideally, it is desirable for the labeling/feature extraction stage to keep pace wi th the vectors of edge points generated by the object segmentation stage. If this was assured all the time, there would be no need for modification of the threshold that has been established. In this case the threshold applied would be precisely the recursive edge value threshold described in the previous section. When the queue becomes too large, the labeling/feature extraction stage should scale up the calculated threshold Te(i) to generate a modified threshold Tm(i) given by ( 3) A multiplicative modification of the threshold has been chosen over an additive one so that it will always result in the same percentage change in the threshold independent of the level. The dependency of the threshold multiplier on the processing queue depth is shown in Fig. 2. As shown in Fig. 2, the object labeller/feature extraction will only affect a threshold change when the queue exceeds a predetermined breakpoint. In this way, the segmentation stage is given ample opportuni ty to ad just the threshold by its local scheme. However, as the queue depth exceeds nl and climbs toward N the labeling/feature extraction stage will increasingly scale up the threshold and throttle the interval generation. PERFORMANCE OF THE SCHEME The performance of the autonomous threshold selection algorithm is demonstrated by the simulation results in Fig. 3.1 3.3. Figure 3.1 shows the image of a hot steel slab surface. In Fig. 3.2, a fixed threshold has been applied following edge enhancement. This fixed threshold was manually optimized for its desirable performance on the image. The results in Fig. 3.3 were derived by applying the adaptive thresholding algorithm described herein. This clearly demonstrates the improvement provided by the adaptive threshold. It is certainly true that a manually selected fixed threshold can, in general, be found that would provide adequate performance on a particular

R. A. Fundakowski et al.

214

image or part of an image. However, experiments conducted on the slab imagery have shown variations of as much as 300 percent in that satisfactory threshold, because of normal variations in the slab surface. The adaptive threshold algorithm that has been developed offers a tractable solution to this problem.

ACKNOWLEDGEMENTS This work was partially supported by the Department of Energy under Contract DE-FC07-79CS40242. The assistance provided by Ms. Vicki Van Slyke and Ms. Sandy Truehart in preparing the manuscript is gratefully acknowledged. REFERENCES

CONCLUSION Adaptive thresholding has been shown to be a practical method of regulating the segmentor performance in conjunction with meeting the throughput constraints posed by a real-time image processing system. The scheme described in this paper performs a key role in the on-line inspection system for hot steel slabs by adapting to the local slab surface characteristics. With the availability of greater processing speed, alternate methods that might also be able to look ahead would likely offer significant advantages.

Suresh, B.R., T.S. Levitt and R.A. Fundakowski (1981) . Real-time automated inspection of hot steel slabs. Structural and Syntactic Pattern Recognition Workshop. Saratoga Springs, New York. June 1981. Suresh, B.R., R.A. Fundakowski, T.S. Levitt, J.E. Overland, T.L. Beckering and T.M. Wi ttenburg (1982) . The automated surface inspection of hot steel slabs. International Workshop on Industrial . Applications of Machine Vision. Research Triangle Park, N.C. May 3-5, 1982.

r-----------------.

SLAB IMAGERY

I I II I

I I I

...

L __

I I

ARITHMETIC PROCESSING UNIT (APU)

CENTRAL SIGNAL PROCESSING UNIT (CSPU)

EOGE ENHANCEMENT EOGE THRESHOLOING AND INTERVAL EXTRACTION

INTERVAL TRACKING LABELING ANO FEATURE EXTRACTION

~P~A!!A~O~SS~

~

OBJECT CLASSIFIER ANO REPORT GENERATION

r

I I

I I ____ ____ J

IDENTIFIED IMPERFECTIONS

HOST COMPUTER

Fig. 1.

I I

Image processing system for slab inspection.

Adaptive Threshold Estimation

215

N

'" ...:::; I

Cl:

w

~

=>

:::E

Cl

-'

Cl % Cl)

w

Cl: %

I-

11------.,.......Fig. 3.1.

Or ig inal image of slab surface with longitudinal face crack.

Fig. 3.3.

Results of thresholding following enhancement.

N PROCESSING QUEUE OEPTH (SCAN LINES)

Fig. 2 ••

Dependence multiplier backlog.

of on

threshold processor

Fig. 3.2.

Results of fixed threshold application following edge enhancement.

adaptive algorithm edge