Knowledge-Based Inspection of Electric Lamp Caps

Copyright © IFAC 12th Triennial World Congress, Sydney, Australia, 1993

KNOWLEDGE-BASED INSPECTION OF ELECTRIC LAMP CAPS A.D.H. Thomas, M.G. Rood, F. Deravl and Y. Chuang Real-Timl! AI Group. DepartmenJ o/Electrical Engineering. University o/Wales. Swansea. UK

Abstract. It is vitally important that the caps of electric lamps undergo visual inspection before they reach the market. This paper describes an on-line, real-time automated visual inspection system which can detect all possible faults. Fault classification is performed by a rule-based system, inferring the result of the inspection from feature values. A special illumination arrangement was developed to reveal the faults. Key Words. Image processing; expert systems; knowledge engineering; artificial intelligence; digital signal processing

1. INTRODUCTION Within the lighting industry, it is realised that potentially lethal GLS lamps are very occasionally Although the electric filament lamp was developed produced. If a diagram of the production line, shown over a century ago, t.he general lighting service (GLS) in Fig. 2, is considered, a danger can arise from any incandescent lamp is still unchallenged as the main lead wire which is not adequately cut off and is left source of domestic lighting (Woodward, 1972). Fig- to protrude through the soldered pads. If such a ure 1 shows a cross-section of a GLS lamp revealing lamp were to be plugged in by a consumer, this the tungsten filament attached to lead wires which wire could bend over to touch the metal case of the are, in turn, connected to brass soldered pads. These cap, resulting in the cap becoming electrically live - a very dangerous situation. soldered pads are surrounded by black "vitrite" a hard, ceramic material. Numerous attempts have been made to detect these uncut wires, including electrical detection, but a CLASS Rl;L.8 fool proof met hod has not yet been developed (for all example of automatic solder joint inspection, see (Bartlett et al., 1988)).

1.1. The problem

MOLYBDENUM FIlAMENT SUPPORTS

In order to reduce the threat, extensive manual inspection procedures are required. For a typical plant, this requires the inspection of over a million lamps per week - a time-consuming and errorprone operation.

Fig. I. Cross-section of a GLS lamp.

Also, in addition to uncut wires (Figure 3a), inspectors have to check for holes in the solder and partial soldering, specified as less than 70% coverage tFigure 3b).

711

This paper will describe an automated visual inspection system which has been developed to identify all faults in GLS lamp caps. Following extensive prototype on-line testing, the system is in the process of being permanently installed in a major UK lighting plant.

ELECTRICAL TESTING

AUTOMATED VJSUA.L INSPECTION

BRUSH

WIRE

(lIaUll'uI and

CUTTING

Machine vision techniques are now well-established in industry as a means of ensuring consistent quality control, often at a speed unmatched by human inspection. However, most of the applications are found in well-defined, restricted situations, often no more than simple measurement tasks. It is possible to construct vision systems for more complex tasks, but these genera.lly have to be tailor-made to suit a particular application. A possible solution to the quest for a more adaptable system arises from the use of knowledge-based techniques. Expert system technology could be used to achieve automated visual inspection in much the same way as an expert system can be used to diagnose human illness, for example. The first step would be to interview the human inspector, discovering the rules for inspection and the relevant image features. The second step would be to develop the knowledge base, probably a series of production rules, which would guide the automated inspection. Image classification would then be achieved by extracting the features from the image under inspection, and passing the features to the expert system which would infer either a "PASS" or a "FAIL" decision.

SOLDERINC

alianl any uncut wine)

Fig. 2. The relevant stages of the production line, showing the position of automated visual inspection. The lamps are processed two at a time, moving right to left.

One of the advantages of this method would be its inherent flexibility - if the system were to be adapted to a different application, only the knowledge base would have to be altered (assuming the range of available features was sufficient). A similar "Generalized Machine Vision Structure" was proposed by Solinsky (1986).

2. ILLUMINATION TECHNIQUES Fig. 3. a) An uncut wire. b) Partial soldering, and a solder hole.

1.2. Automated visual inspection Computer vision is a relatively new discipline, but many useful pattern recognition methods have been developed, including template matching (Stockman and Agrawala, 1977) (and associated Hough transform methods), statistical pattern recognition (Duda and Hart, 1973), and syntactic pattern recognition (Fu, 1982). When computer vision techniques are employed in highly-constrained industrial applications, such as automated visual inspection, they are more commonly referred to as "machine vision".

In any visual inspection problem, the quality of the lighting is paramount. A computer vision system is only as good as the image presented to it. This is especially true of real-time applications where there is little time for sophisticated image enhancement techniques. For this project, the design of the illumination was critical because it had to ensure the visibility of holes and uncut wires. For example, if light is shone directly down a hole, the hole will appear as bright as the solder surrounding it and will no longer be visible to the camera. It was also discovered that the orientation of the uncut wire in relation to the lighting had a strong influence on how visible the

712

wIre was. It was therefore import.ant to ensurt> that the illumination was isotropic. Another requirement of the lighting arrangf'nlf'nt was that it should restrict. any reflective glint ing from the vitrite that forms the majority of the cap. Any bright patches could be mistakenly identified as uncut wires. In order to reduce reflections, the illumination has to be diffuse, and not sharply-directed.

VIDEO CAMERA

The solution was provided by an omnidirectional illumination arrangement (Batchelor, 1985), the crosssection of which is shown in Fig. 4. Diffuse lighting is achieved by reflecting light off the interior of an integrating hemisphere which is pa.inted matt white. Strategically-placed baffles prevent a.ny direct light from reaching the lamp cap under inspection. The circular nat ure of the hemisphere ensures tha.t the illumination is isotropic.

illUMINATION SOURCE

INTEGRATING HEMISPHERE

The inside of the hemisphere is painted so as to allow most reflection from the lower portion. Light is Fig. 4. Cross-sectional side view of the image illumialso reflected off the inside of a white meta.l sleeve nation arrangement showing the omnidirectional lightwhich projects downwards, very close to the cap uning (the dotted lines show the directions of light travel). der inspection. Thus, most of the light hits the la.mp cap at a low angle, and this ensures the visibility of noise. For this reason, the first processing stage is any solder holes. an image-smoothing operation, achieved by digital low-pass filtering. The whole image is convolved with a 7 x 7 kernel in a digital signal processor, the 3. INSPECTION ALGORITH~lS Plessey PDSP16488. The inspection algorithms have to be highly efficient due to the speed of the production line one bulb is produced every half-second. Greatest speed is required at the front-end of the processing, as raw images contain a massive amount of data. Hence, dedicated hardware is employed at this front-end. At the later, higher-level stages of processing, e.g., scene cla..,sification, the data volume is much reduced and more flexible algorithms are required. Hence, a genera'l-purpose, parallelprocessing system is employed for the higher-level operations, in this specific case, this is based on Transputers.

3.1. Front-end processing The purpose of front-end processing is to reduce the input image data to a manageable amount in as short a time as possible. However, this ciesign for speed results in the algorithms being sensitivl:' to

Unfortunately, a sharp cut-off in the spatial frequency domain for the low-pass filter leads to slight blurring in the spatial domain. Other filters, such as the median filter, would avoid blurring but can not be implemented in the digital signal processor. Fortunately, thp. blurring does not significantly affect the later stages of processing.

3.2. Image segmentation The second stage is to segment objects in the image from the background. Due to the highly-restricted range of objects and lighting involved, it is possible to achieve this segmentation by means of a thresholding operation. Pixels are assigned to foreground or background regions on the basis of the illumination intensity of ea.ch individual pixeJ. Automatic determination of the optimum threshold level was considered, but was found to be too slow. Hence, a global threshold, which can be varied by the user,

713

is applied. Location of objocl ccntroids

Boundary pattern analysis is employed, as this technique requires considerably reduced computation as compared to binary shape analysis. This is because the number of pixels on an object's boundary is usually far fewer than the number of pixels contained within the area of that object. Before boundary pattern analysis can be used, the boundaries of all the objects present in t he image have to be tracked. Care must be taken when applying boundary-tracking algorithms to ensure they will function correctly, no matter what the shape of the object. The following strategy guarantees correct boundary tracking of all shapes (Davies, 1990):

Fercl diamelal

0

040

1. Track round each boundary, consistent.ly keeping to the left path. 2. Stop the tracking procedure only when passing through the starting point in the original direction. The result of the boundary-tracking operation is that all of the objects present in the image are converted into individual (x, y) coordinate codes.

ContainmCIIl

Fig. 5. Some of the features which are derived from the boundaries of the objects.

feature is containment, indicating which objects are fully contained within others: this was found to be useful for checking for holes.

4.2. Expert systems

4. RULE-BASED CLASSIFICATION 4.1. Feature. extraction For reasons of computational complexity, classification is limited to boundary pattern analysis. Hpnce, the only features that can be derived must be calculated from the boundaries of objects; however, this is sufficient for dealing with a vast range of industrial inspection problems. The coordinate codes for each object are examirH'd and a range of geometrical features are calculat.f'd. The features with the greatest discriminating ability must, of course, be selected so as to aid the classification process. These include some surprisingly simple features such as object perimeters and location of object centroids. Some of these features are shown in Fig. 5. Other features include the Feret diameters, which are the measurements that would be obtainpd by using calipers on the object, and by taking Feret diameters in two perpendicular directions it is possible to obtain a value for aspect ratio. Another

Expf'rt systems introduce the possibility of capturing human expert.ise on a computer. The most common implementation of expert systems is in the form of a "shell" program, which is divided into two sections: a knowledge base and an inference engine. The knowledge base is the data set in which the expertise is stored, and the inference engine is capable of logically inferring useful conclusions from this knowledge. The inference engine will generally not vary between applications, whereas a different kno\\'lf'dge base will usually have to be constructed for each application. Hence, the first step in producing an expert system is to formulate the knowledge.

In this present work, the lamp cap inspectors and factory engineers were interviewed, and the knowledge obtained was encoded in the knowledge base of a commercially-available expert system shell. The selected shell, LEONARDO (1993), has facilities for encoding knowledge as frames and production rules. The rule base, containing the inspection knowledge, was developed and tested off-line in a laboratory. However, the processing speed of such a shell is not sufficient, and its operation is not deterministic enough, to be used in the real-time lamp cap

714

inspection (Jambunathan et al., 1991). Thus, the rules have to be re-coded as explicit C functions for execution on the Transputer network. As a result, a processing speed of 0.4 seconds for an image of two lamp caps can be achieved, which is more than sufficient for correct on-line application on the current production facility.

Initial trials of the automated lamp cap inspection system have been successful. Work is underway to improve the long-term reliability of the system before it is fully installed on-line.

5. TESTING

the ACME Directorate of the SERC and GE Lighting Ltd.

is t.he possibility of introducing some form of supervised machine learning to automate this step.

Acknowledgements -

Whilst on-line testing is still at an early stage, initial results from short-term (3 month) tests seem promising. Tests have been performed in which the classification performance of the system was compared against that of a human inspector (assuming the judgement of the inspector is always correct). In the test, over three and a half thousand lamps were considered. By far the majority of the lamps, over 99%, were fault-free and were correctly judged as such by the aut.omated system. Six cases of partial soldering, and four cases of holes in the solder arose. These were all correct ly identified by the s)·st.em. There are two possible classification errors that can occur: a false negative error would occur when a faulty lamp was signalled fault-free, and a false positive error would occur when a fault-free lamp was signalled as faulty. The automated inspection system classified 19 good. lamps as faulty. This represented a false positive error rate of less than 0.01%. No faulty lamps were classified as fault-free, giving a zero false negative error rate. This is very important, as it shows that no potentially dangerous lamps would reach the consumer.

The authors would like to thank

for thp.ir support and funding throughout this project.

7. REFERENCES Bartlett S.L., Besl P.J., Cole C.L., Jain R., Mukherjee D. and Skifstad K.D. (1988). Automatic solder joint inspection. IEEE Trans. on Pattern Analysis and Machine Intelligence, 10,

31-43. Batchelor B.G. (1985). Lighting and viewing techniques. In: Automated Visual Inspection (Batch· elor B.G., Hill D.A. and Rodgson D.C., Eds.), pp. 103-179. Elsevier, Amsterdam. Davies, E.R. (1990).

Machine Vision: Theory, Algorithms, Practicalities. Academic Press,

London. Duda R.O. and Rart P. E. (1973). Pattern Classification and Scene Analysis. Wiley, New York.

Fu K.S. (1982). Syntactic Pattern Recognition and Applications. Prentice-Hall, New Jersey. Jambunathan K., Lai E., Rartle S.L. and Button B.L. (1991). Development of an intelligent front end: an experience. Eng. App. of Artificial Intelligence, 4, 385-392. LEONARDO (1993). Creative Logic, Uxbridge.

6. CONCLUSIONS Most industrial vision systems are tailor-made to specific applications. However, by separating the inspection procedure into two parts, feature extraction and rule-based classification, a general principle has been demonstrated: that rule-bllSed met.hads can be used to produce more flexihle automated visual inspection systems.

Solinsky J.C. (1986). The use of expert systems in machine vision recognition. In: Vision '86 Conference Proceedings, pp. 139-158, Detroit. Stockman G.C. and Agrawala A.K. (1977). Equivalence of Rough curve detection to template matching. Gomm. of the AGM 20,820-822. Woodward F. Incandescent lamps. In: Lamps and Lighting (Renderson S.T. and Marsden A.M., Eds.), pp. 172-191. Edward Arnold, London.

At the moment, the knowleJge base has to be constructed by an experienced programmer, but there

715