Using machine vision for the inspection of industrial assemblies

Using machine vision for the inspection of industrial assemblies C. Tsatsoulis

and King-sun

Fu

A data-driven, general method for visual inspection of industrial assemblies and its implementation as a high level command translator is presented. The system is based on measurement and computation of generic assembly characteristics and comparisons of these characteristics to known values and relations. The application of the system to fairly complicated sub-assemblies is included as an example. Keywords: visual inspection, industrial assemblies, computer programs

Reference 23 the system proposed by him was capable of inspecting only circularly symmetric assemblies. In Reference 24 he proposed the INSPECTOR, a computer vision inspection system which can learn to classify patterns by being shown several identified good and bad parts. The system forms a model of the part during the training session, containing a set of identifying points used for locating parts, a set of inspection regions and a set of inspection tests. For a comparison of the registered images the system determines regions of the picture that differ significantly from the model image and then determines the test parameters needed to allow it to distinguish between good parts and bad parts with an arbitrary number of defects.

In the past few years the interest in advanced automated manufacturing and production has grown dramatically. Attention has been focused on CAD/CAM, robotic and advanced computer systems, including automatic design and testing systems, and expert systems for manufacturing. Although the full automated ‘factory of the future’ is still a long-term goal, many ofthe obstacles in the way of accomplishing this objective have been overcome, or at least identified: design, manufacturing and assembly have all been partially automated. A very important step ofproduction, though, has been mainly ignored: assembly inspection. It is true that testing of a finished product will probably always be necessary to verify product performance specifications, but on-line testing of the assembly steps is equally important.

In this paper a method for visual inspection of industrial assemblies will be presented. Some preliminary work was presented in short form in Reference 25 and was based on the initial research presented in Reference 26. Here, both the principles behind the proposed technique and an implementation of the inspection system on a fairly complicated assembly will be presented.

Today product inspection and testing is primarily based on functional techniques, such as electrical, mechanical and hydraulic. Visual inspection has been limited to printed circuit boards (PCBs), and - to a lesser extent to intergrated circuits (ICs). Generally speaking, the visual inspection techniques found in today’s literature fall into three major categories: pixel-by-pixel comparison or with some reference pattern; feature comparison inspection; generic property veriticationl’l. Some examples include the work by Mountjoy et a1[‘1,who have described an experiment to detect missing circuit components using pixel-by-pixel comparison. Ito131 has developed a colour effect method, essentially a pixel-by-pixel comparison technique, for detecting small defects in LSI mask patterns. Goto et a/l41have designed an IC mask pattern inspection system that uses a hierarchical detection algorithm. Hsieh and Fu151proposed an automatic visual inspection system for IC chips by dividing each chip into n mask sub-patterns and using syntactic pattern recognition techniques. Considerable work has also been done in the area of PCB inspection and automatic IC lead bonding16-“1.

Objectives Our primary goal was to design a realistic system that would specialize in visually inspecting industrial assemblies. As a ‘realistic’ system we understood one that would be only the first step of an advanced, fully automated system and, at the same time, would also be complete enough to be immediately applicable. A ‘realistic’ system would be one that would not need sophisticated, expensive hardware, but would also not trade off price and convenience for execution speed and storage space. It would be a system that would not only be able to perform satisfactorily in a controlled laboratory environment. but also be able to handle the hazards (changes in lighting conditions. changing part position and orientation, noise, etc) of a production line. Finally, it would be a system that requires minimum training and computer knowledge for operation, and is also flexible and powerful enough to accommodate small batch production by being able to be trained to inspect new assemblies in short times.

Vision has also been widely used in assisting robots in automated assemblies by recognizing and locating parts, inspecting single parts for defects and determining paths for the robot arm movementsl’*-**I. On the other hand, assembly inspection using visual digital techniques has been limited. Significant is the work by Perkins123,241. In 0308-9126/86/040263-08 NDT INTERNATIONAL.

VOL 19. NO 4. AUGUST

$03.00 1986

0

1986

Butterworth

& Co (Publishers)

Ltd 263

~-~

v2[

v, i l

',

"1

Part~A

assembly step we can assume that the previous step was completed correctly. This logical assumption introduces constants in the assembly scene, which can be used, together with the variables introduced by the new parts to be assembled, to reach a classification decision[25'261.

iv3

I

=', I

I

@

Sub-assembly (i-1)

Sub-as~mbly

i

Fig. 1 Exampleof inspection of two sub-assemblies Our objective was to satisfy as many of the above goals as possible and - - as will be shown - - this has been accomplished• It has to be noted that we did not concern ourselves with tolerance problems (these are discussed in more detail later on). The primary goal here was to detect the relatively big errors and classify them according to error type, so as to allow possible corrections. As mentioned in the introduction, visual inspection cannot always be used for complete testing, but it can decrease "obvious' assembly errors, reduce rejects and relax otherwise stringent component specifications, thus dramatically increasing productivity. The recognition of these 'obvious' errors was our aim.

The

inspection

A simple example of the above technique is shown in Figure 1. Let us assume that the assembly step i is to be inspected. This implies that the assembly step ( i - l ) has been completed and already inspected. We can then assume that sub-assembly (i-1) is correct and use its geometric characteristics as constants. In Figure 1cl is the height of sub-assembly ( i - 1), c2 its diameter, c3 its centre and c4 the diameter of the inner hole. Let us assume that assembly step i consists of inserting part A into the hole of sub-assembly ( i - l ) , resulting in sub-assembly i. Then vl will be the maximum height of sub-assembly i and a variable, since it depends on the degree of insertion of part A (v2 and v3 can also be variables if verticality is required). So, assuming that our test was simply a perfect insertion inspection, we could use vl/cl as our inspection criterion. This value will be independent of position and size of the assembly and (as will be shown later) independent of most of the sub-assembly orientations and of moderate amounts of noise. Constant scene characteristics can also be used to calculate other constant values in order to save execution time. So, in the above example, any of the constants could have been used to determine all others (with the exception of the centre location). The method presented above greatly simplifies the inspection task and the training necessary, since by using generic values no time and space consuming training

method

The method was based on the observation that for an

SAIL rI

Assembly scene

t

I

User commands

'1 !-I I

SAI L program

SA, L

I

compiler

1

C program ~ I

I I I

Inspection r

1

I I I

I I I

,, ' 'mage H i ~1 I

processing

I I I L_

'

assembly perimeter

HMeasurementan H I ' t I calculation of characteristics

.

relations

Testing

•

I

Decision

I I J

Test parameters

Fig. 2 Blockdiagram of the inspection method 264

NDT INTERNATIONAL. AUGUST 1 9 8 6

sessions and sample images are needed. It also helps solve the major problem of having to inspect numerous, sometimes completely different, sub-assemblies during one assembly inspection.

~"o, ~,j~..~ (~ ~ (]~=.^ ~4;~jr~-~.7~, ,l~,U\ ~l,z--~~i

/

/Screw Torsion blades

The algorithm used to implement this inspection technique is shown in Figure 2. To avoid long execution times thresholding was used in most cases to separate the assembly perimeter from the background. Next, points on the assembly perimeter were established and from these points constant and variable assembly characteristics were calculated. Then, all these values were used to reach a classification decision by comparing them with test values stored in a test database. Initially the thresholded binary pictures were stored as two-dimensional integer arrays, with one pixel occupying two bytes (16 bits on the VAX). A storage space saving of almost 88% for a 256 × 240 image (64 440 pixels) was achieved by storing each pixel as a bit having value 0 for black and 1 for white. By using a floating list pointer the pixel retrieval times showed no change between the bit storage and the byte storage method.

SAIL - - an inspection c o m m a n d translator After the algorithm described earlier was completed and tested on some sub-assemblies, a small inspection subroutine library emerged. It was at this point that a decision had to be made: should we leave the system in subroutines that could be called from programs written by the user(s), or should we further simplify the inspection process? Trying to comply with the 'realistic' system requirement we developed simple assembly inspection language (SAIL). SAIL is a C-language based high level command translator and language. It allows the writing of short and simple programs that perform visual inspection of assemblies. Their input is an image file of any dimension and grey level scale, and the output a (possibly multiclass) classification of the assembly. The advantages of SAIL are the ease of programming, since virtually no previous knowledge of a programming language is needed, the power of the inspection algorithms available and, most important, its expandability. The SAIL commands can be divided into three major categories: control symbols, dot commands and subroutines. The appendix gives an introduction to SAIL and describes these three commands in detail.

An example - inspecting the assembly of the model HGE pow e r steering valve The assembly of the HGE Hydraguide power steering valve (pictured in Figure 3) was chosen as an example of the proposed inspection system, since it is a fairly complicated assembly, but it can also be easily manually assembled, which made it practical*. Two basic assumptions were used for the inspection: that there are two possible camera positions, and that the robot or other assembly machine will be positioning perfect parts at least in the approximate correct area. In most cases the only special lighting condition needed was backlighting. Since the system uses thresholding and the lighting conditions were not constant throughout the inspection, a program determining the threshold each

Fig. 3 HGE exploded view (from HGE Service Manual, TRW Ross Gear Division, HGE-I02, 1.5M 3/81 CPC)

time would be necessary. For simplicity though, the thresholds were predetermined. There was no need for exact orientation or position of the parts, as long as they were well focused inside the camera view and big enough to be clearly visible. An effort was made to make the inspection reliability as high as possible. So, the various classification criteria were chosen such as to detect all the incorrect assemblies, even if that meant misclassifying some of the correct assemblies. Before the inspection, the geometrical characteristics of the parts used in the assembly were measured (information that could also be acquired from drawings or the design database, in general) in order that the relations between these values be used to calculate some of the constant assembly characteristics given some other, measured, constant ones, and also be used as the inspection criteria. The equipment used were a General Electric TN 2500 CID camera, a Grinnell digitizer and standard back (and overhead) lights. All software was written in C and run on a DEC VAX 11/780 computer on the UNIX** operating system. The images manipulated were 256 × 240 pixels, 8 bit grey scale. Figures 4-9 show some of the subassembly configurations inspected t . The inspection tasks can be classified into roughly three categories: part positioning and orientation, hole alignment and plastic and metal seal positioning. To give a better understanding of how the system works an example will be given from each category. In figures 4 and 5 a complex position and orientation problem is demonstrated. Part A (screw) is the constant in the scene and part B (torsion blades) is the variable, since it is its insertion we try to establish. Figure 5 shows the thresholded image and the trace lines of the characteristics measured. The system decided in this case that the blades were not centred to the right and were not inserted **UNIX is a registered trademark of Bell Laboratories.

*The assembly steps were according to the Hydraguide Hydrostatic Steering System HGE Service Manual, TRW Ross Gear Division, HGE-lO2, 1.5M 3/81 CPC. Hydraguide is a trademark.

NDT INTERNATIONAL . AUGUST 1986

1"Thespiral pixel printing package used to print these images was developed at the School of Electrical Engineering, Purdue University, by Atte Kortekangas

265

Table 1. SAIL inspection program for the position md orientation problem

$ Check for the centring and flush insertion of the torsion blades $ dimension(256,240); threshold( 120); $ find top of blades $ linesearch ( 1,1,20,256,20,120,x_top,y_top); .t ystart<0; exit; $ blades missing, exit $ .t crvtrace(60,x-top, y-top, x_mid, y_mid); $ lower point of blade $ .t (x_top > x_mid+5)l I (x_top < x_mid-5); $ check verticality $ .t $ Find blade width and establish a right blade point $ l i nemeasu re ( 1,1 ,x_m id ,y_m id,x_rig ht, N U L L,width ); .a

increment=width/2; Fig. 4

.a

Incorrect third sub-assembly

flush at the bottom of the screw. The SAIL version of the inspection program is presented in Table 1. Figures 6 and 7 show a hole alignment problem. The part (rotor) constitutes the constant in the scene, while the area and/or presence of the holes the variables. Figure 7 shows the thresholded image with the area of the first three holes measured in grey. The system found the centre and radius of the rotor (constant characteristics) and from those calculated the radius and area of a perfectly aligned hole. It then measured the area of the holes (variable characteristic), compared them with the previously calculated value and made its decision. Table 2 shows the SAIL version of this inspection program.

$ Find left, centre and right heights $ linemeasu re(3,1 ,x_mid+5,y_top, N U LL,NU LL, left_height); linemeasu re(3,1 ,x_mid+increment,y_top, N U LL, N U LL,centre_height); linemeasure(3,1 ,x_right-5,y_top,N U LL,N U LL, right_height); $ NOTE: NULL is used as a dummy variable $ $ Check for blade centring $ .t (left_height>centre_heig ht+5)l I (right_height> centre_height+5) (left_height>right_height+20)l I(left_height> right_height+20); .t .a

In Figures 8 and 9 the detail of two sub-assemblies with a plastic seal positioning inspection task are shown (correct-incorrect). The system used histogram equalization to enhance the initial image quality and again

insertion_test=centre_height/width; .a

$ Check for flush insertion $ .t insertion_test>8.33; .t 8

found the centre and radius of the centre hole (constant characteristics). From these values it calculated a search path and tried to establish the seal positioning criterion. In this particular example the system was able to reach only a ~yes/no" decision, particularly due to the bad image quality and its reliance on thresholding. Table 3 lists the SAIL version for this inspection program.

Fig. 5 Thresholded image of Figure 4

266

Testing of the above method showed that it reached 100% success in multiclass classification of assembly errors for 12 out of the 24 sub-assemblies it was presented with, gave minor misclassification errors for 1 out of 24 subassemblies and was capable of'yes/no' answer for 3 out of 24 sub-assemblies. Eight sub-assemblies could not be inspected due to the part size (radii between 3 and 7 ram) and the camera resolution. Since the initial goal was to inspect and detect obvious errors it can be said that it has been satisfied.

NDT INTERNATIONAL. AUGUST 1986

Table 2. SAIL inspection problem for the hole alignment problem $ Inspect the hole alignment of the rotor and the rotor shaft and manifold $ dimension (256,240); threshold (80); find a perimeter point of the part $ linesearch (1,1,40,180,30,110,x,y); circle (4,0,2,x,y,x_centre,y_centre, radius); $ centre, radius $ $ Compute needed test parameters $ .a

hole_radius = 0.061 *radius; test_area = 3.14*hole_radius*hole_radius; s_radius = 0.8*radius; .a

$ The following demonstrates how iteration can be used in SAIL $ $ Define user variables $ .V

int _i; .V .W

for (_i = 0; _i<7; _i+ +)

{

Fig. 6

.W

Incorrect seventh sub-assembly

Tolerances were of no major importance, since again, it was the relatively large errors we were trying to detect and classify. Since the part size alone introduces some error, and the size of the parts was relatively small (the biggest one having a 35 m m radius), it can be said very roughly that the measurement error averaged 5-10 pixels, which (in a 256 x 240 image), is about 5% in the worst case. (This was established c o m p a r i n g the thresholded image with an image where an edge detector was applied.) For the

$ Locate hole ('cnt' remembers the previous angle position) $ crclsearch(1,-1,0,cnt,x_centre, y_centre, s_radius, x_hole, y_hole); $ Measure the area of the hole $ area (x_hole, y_hole, value); .t (value180); $ check hole area $ .t .w

}

.W

Et

images used (256 x 240 pixels, 8 bit grey level), the execution times on a D E C VAX 11/780 computer ranged between 0.7 and 8.0 s, averaging 3.0 s per inspection.

Conclusions and future w o r k A simple but powerful method for visually inspecting industrial assemblies and its implementation as the high level c o m m a n d translator SAIL have been presented. The power and applicability of SAIL are based on the high performance of its basic sub-routines in inspection programs, the ease and simplicity of its structure and syntax and the virtually unlimited expandability. Using SAIL, short programs for visual inspection can be written and easily changed to a c c o m m o d a t e updating o f the design database.

Fig. 7

Detail of thresholded image of Figure 6

NDT INTERNATIONAL . AUGUST 1986

The system showed great tolerance to noise and required no exact positioning or orientation of parts. Since it bases its decision on measurement of various characteristics it is possible that, for some part, measurement of a particular characteristic is not possible for a certain viewing angle. This problem was not encountered in the assemblies that were tested, but could be easily solved. Execution times

267

Table 3. SAIL )ositioning task

inspection

program

for

seal

$ Inspect the correct positioning of the plastic seal on the outer housing cover $ dimension (256, 240); enhance (1); threshold (120); Two separate images are created, one enhanced, one binary $ linesearch(1,1,40,180,30,1 lO,x,y); $ locate centre hole $ circle(4,0,2,x,y,x_centre, y_ centre, radius); $ find centre, radius $ .a

search_radius=3.95*radius; .a

$ Count seal points $ crclmeasure(-1, - 1 , x_ centre, y_ centre, search_ radius, 15, seal_ points); .t seal_ points >-- 90; .t Fig. 8

Detail of correct eleventh sub-assembly

were satisfactory but could be greatly reduced if smaller images were inspected. For the above example 64 x 60 images were initially used, averaging 0.4 s for inspection. For assemblies though that need detailed examination a better image resolution may be required. A very important step has been made in addressing the inspection of industrial assemblies, but to achieve full automation of the visual inspection process many more problems have to be solved. The most important one is the design of an assembly database, which, together with the part database (eg CSG model), will be used by robots and

other positioning devices for assembly. A direct link to such a database will allow automatic programming of the inspection procedures and automatic modification of them in cases of design changes.

Acknowledgements This work was supported by NSE Grant ECS 81-19886. This work was also supported in part by CIDMAC, a research unit of Purdue University, sponsored by Purdue, Cincinnati Milacron Corporation, Control Data Corporation, Cummins Engine Company, Ransburg Corporation and TRW.

Appendix. Introduction commands

to

SAIL

and

its

SAIL, being C-language based, allows variables that have to start with a letter or an underscore. The variable length is internally limited to 25 characters and the total number of variables is limited to 100, This, though, should not present a problem, since rarely will there be an application where more than 100 variables are needed at the same time, and the system provides means from removing no more useful variables from the variables' stack. The translator saves each variable name on this variables' stack and renames it as an element of its internal OBJARRAY. Variables starting with an underscore are recognized as user-defined variables and are not translated, but are still saved on the variables" stack.

Fig. 9

268

Detail of incorrect eleventh sub-assembly

As noted in Table 4 some sub-routine arguments demand a specific data type. This is since they are option values (if they are restricted to integers) or are used to return some value (if they are restricted to variables). An important idiosyncrasy of SAIL is that it allows only integers inside sub-routines for reasons of internal organization. This

NDT INTERNATIONAL. AUGUST 1986

Table 4.

Sub-routine commands of SAIL

#: argument must be integer X: argument must be variable *: non-restricted argument area (*,*,X) call (X) circle (#,#,#,*,*,X,X,X) crclmeasure (#,#,*,*,*,*,X) crclsearch (#,#,#,X,*,*,*,X,X) crvtrace (#,*,*,X,X) dimension (#,#) enhance (#) linemeasure (#,#,*,*,*,*,X) linesearch (#,#,X,*,*,*,*,X,X) remove (X) threshold (#)

can be easily solved by the arithmetic a s s i g n m e n t operators which accept d e c i m a l numbers. SAIL checks for incorrect syntax, u n k n o w n system functions a n d c o m m a n d s , syntax errors o f a r i t h m e t i c a n d logical operations, usage of u n k n o w n variables, usage o f incorrect sub-routine arguments, various overflows, etc. In general, most errors will be detected by SAIL, before the C c o m p i l e r takes over. Of course, errors in the u s e r - p r o v i d e d code c a n n o t be determined. A short description of the SAIL c o m m a n d s follows, with some e x a m p l e s o f the option values in sub-routines.

Control symbols.

T h e control s y m b o l s a r e ; (end o f c o m m a n d line), & (end of inspection p r o g r a m ) a n d $ ( c h a r a c t e r enclosing c o m m a n d s ) .

Dot commands. These c o m m a n d s are o f the form • {COMMANDS} • where < l e t t e r > c a n be any of {a,r,t,s,v,e,w}. U s i n g the dot c o m m a n d s we can assign values to variables, c h a n g e set register values, perform tests a n d a d d user defined sub-routines a n d variables. So, <.a> is the a r i t h m e t i c a s s i g n m e n t dot c o m m a n d . W i t h the

Table 5.

Registers of SAIL

Number

Parameter controlled

1 2 3 4 5 6 7 8 9

horizontal starting position horizontal ending position vertical starting position vertical ending position direction of linear movement direction of circular movement increment of linear movement (in pixels) increment of circular movement (in rad) binary value of thresholded background and assembly parts printing of detailed syntax error messages

a

(on/off)

NDT INTERNATIONAL . AUGUST 1986

register dot c o m m a n d <.r> the value of the registers shown in T a b l e 5 can be changed• The dot c o m m a n d <.t> helps provide logical o p e r a t i o n s helpful for testing. Together with it comes the c o m m a n d exit that allows i m m e d i a t e t e r m i n a t i o n o f the inspection in cases of t e r m i n a l errors. The c o m m a n d s <.s, .v, .e, .w > allow inclusion o f user-defined sub-routines, definition o f user variables, definition o f user external variables a n d inclusion of user-written code in the m a i n p r o g r a m body, respectively. F o r all dot c o m m a n d s the syntax is simple a n d b a s e d on c o m m o n sense. So, for example, an error message will be given if a v a r i a b l e is assigned the value o f a previously u n d e f i n e d variable, or if a test is p e r f o r m e d on an u n k n o w n variable. Errors will also be p r o d u c e d by assigning values to non-existent registers, or by expressions o f the form x + y = 5.

Sub-routines. Sub-routines a n d their allowed a r g u m e n t s are shown in T a b l e 4. A short d e s c r i p t i o n o f each one follows: area - - measures the area of possibly a hole. The first two p a r a m e t e r s are the x a n d y positions o f the starting pixel. The third one returns the area m e a s u r e m e n t . e n h a n c e - - does i m a g e e n h a n c e m e n t with the type d e p e n d i n g on the p a r a m e t e r : h i s t o g r a m e q u a l i z a t i o n (1), r a m p (2), e n h a n c e contrast (3). call - - calls a user-defined sub-routine. This saves the user from using the c o m m a n d .w for a single line. circle - - this s u b - r o u t i n e finds the centre a n d radius o f a part. T h e first a r g u m e n t indicates the type of m e t h o d desired (0: m a x i m u m arc with correction, 1: m a x i m u m arc without correction, 2: p e r p e n d i c u l a r s , 3: three points with curvetrace, 4: three points without curvetracc, negative: register value) (see Reference 26 for m o r e details). The second a r g u m e n t gives the pixel value o f the object (0: black, 1: white), the third one the possible n u m b e r o f holes that can be encountered, the fourth a n d fifth ones the x and y values o f the starting point a n d the sixth, seventh a n d eighth return the x,y position of the centre a n d the radius o f the object. crclsearch - - given the centre a n d radius o f an object a n d a search radius, this sub-routine locates specific characteristics (grooves, holes, etc). c r c l m e a s u r e - - given the centre a n d radius of an object a n d a search radius, this sub-routine m e a s u r e s specific characteristics present (seal points, grooves, holes, etc). crvtrace - - performs curve tracing, to help find angles of orientation, p e r i m e t e r points, ew. d i m e n s i o n s - - helps the user define the x a n d y d i m e n s i o n s o f the image inspected. l i n e m e a s u r e - - measures the length o f an object given a starting point a n d m e a s u r i n g directions. linesearch - - s i m i l a r to crclsearch, only s e a r c h i n g on a line. remove - - removes the variable i n d i c a t e d from the variables" stack. t h r e s h o l d - - given a positive value as a r g u m e n t it will

269

threshold the image using this value, otherwise it will determine the threshold by itself. The latter is time consuming and can be easily avoided in controlled testing environments.

12

T h e p o w e r o f S A I L lays in two facts: first, the c o m m a n d s i m p l e m e n t e d w e r e a l m o s t the o n l y o n e s n e e d e d d u r i n g the testing sessions, a n d second, it allows u n l i m i t e d expansion, by e n a b l i n g insertion o f user-defined variables a n d code. In o t h e r words, S A I L c a n be u s e d e i t h e r as a virtual a s s e m b l y l a n g u a g e or as C - l a n g u a g e s u b - r o u t i n e s i n s i d e larger p r o g r a m s .

14

13

15 16 17

References 1 Fu, K.S. 'Robot vision for machine part recognition'SP1E Conf 442 Robotics and Robot Sensing Systems. San Diego. CA (August 1983) pp 2-14 2 Mountjoy, D. et al 'Automatic visual inspection of printed circuit boards - is the problem too difficult?" Proc Seventh Annual Automatic Image~ Pattern Recognition Syrup (May 1977) pp 152-167 3 lto, T. "Pattern classification by color effect method'Proc Second lnt Joint Conf on Pattern Recognition (August 1974) pp 76-77 4 Goto, N. et al "An automatic inspection system for mask patterns' Proc Fourth lnt Joint Conf on Pattern Recognition (November 1978) pp 970-974 5 Hsieh, Y.Y. and Fu, K .S. 'An automatic visual inspection system for integrated circuit chips' Computer Graphics and hnage Processing 14 (December 1980) pp 293-343 6 Horn, B.K.P. "A problem in computer vision: orienting silicon integrated circuit chips for lead bonding' Computer Graphics and Image Processing 4 (1975) pp 294-303 7 Jarvis, J.F. 'Automatic visual inspection of western electric series 700 connectors" Proc Pattern Recognition and Image Proce3~ing (June 1977) pp 153-159 8 Jarvis, J.F. 'Computer vision experiments on images of wirewrap circuit boards" Proc Workshop on Industrial Applications of Machine Vision. Research Triangle Park, NC (May 1982) pp 144-150 9 MeVey, E.S. and Van Tol, A. "An experimental printed circuit board drilling system automated by pattern recognition'Pattern Recognition 11 4 (August 1979) pp 271-276 10 Sterling, W.M. "Automatic non-reference inspection of printed wiring boa rds" Proc IEEE Conf on Pattern Recognition and Image Processing (1979) pp 93-99 11 Kaufmann, P., Medioni, G. and Nevatia, R. 'Visual inspection using linear features' Proc IEEE Comp Soc Conf Computer Vision and Pattern Recognition (1983) pp 496-497

18 19 20 21 22

23 24 25 26

Agin, G. "Computer vision systems for industrial inspection and assembly' Computer 13 5 (May 1980) pp 11-20 Birk, J.R., Kelly, R.B. and Chen, N. "Visually estimating workpiece pose in a robot hand using the feature points method' Proc IEEE Con..( Decision and Control (1978) pp 1407-1412 Birk, J.R., Kelly, R.B. and Martins, H. 'An orienting robot for feeding workpieces stored in bins" 1EEE Trans SMC S M C - l l (February 1981) pp 151-160 Ferloni, A. 'ORDINATORE: A dedicated robot that orients objects in a predetermined direction' Proc lOth lnt Symp on Industrial Robots (1980) pp 655-658 Koshiken, K. and Nemi, A. 'Object recognition and handling in an industrial robot system with vision" 8th Int Symp on Industrial Robots (1978) pp 744-755 Ray, R. and Wilder, J. 'Robotic acquisition of jumbled parts from bins by visual and tactile sensing'Intelligent Robots: 3rdlnt. Conf. Robot Vision and Sensory Controls Ro ViSeC3. Proc SPIE 449 (1983) pp 37-47 Perkins, W.A. ~Simplificd model-based part locator'Proc 5th lnt Joint Conf on Pattern Recognition (1980) pp 260-263 Saraga, P. and Skoyles, D.R. 'An experimental visuallycontrolled pick and place machine for industry" Proc 3rd lnt Joint Conf on Pattern Recognition (1976) pp 17-21 Taylor, W.K. and Ero, G. "Real-time teaching and recognition system for robot vision' Industrial Robots 7 2 (June 1980) pp 99-106 Ayaehe, N. 'A model-based vision system to identify and locate partially visible industrial parts' Proc IEEE Comp Soc Conf Computer Vision and Pattern Recognition (1983) pp 492-494 Baird, H. and Lurie, M. "Precise robotic assembly using vision in the hand' Intelligent Robots: Proc 3rd Int Conf on Robot Vision and Sensor), Controls (International Society for Optical Engineering, 1983) pp 660-666 Perkins, W.A. "Using circular symmetry and intensity profiles for computer vision inspection" Computer Graphics and Image Processhlg 17 (October 1981) pp 161-172 Perkins, W.A. 'INSPECTOR: A computer vision system that learns to inspect parts" IEEE Trans Pattern Analysis and Machine Intelligence PAMI-5 6 (November 1983) pp 584-592 Tsatsonlis, C. and Fu, K.S. "A computer vision system for assembly inspection' SPlE Conf 521: Intelligent Robots and Computer Vision (November 1984) pp 352-357 Tsatsoulis, C. and Fu, K.S. "A computer vision system for industrial assembly inspection" Report TREE 84-30 (School of Electrical Engineering, Purdue University. 1984)

Authors C. Tsatsoulis is with the School of Electrical Engineering, Purdue University, West Lafayette, Indiana 47907, USA, as was the late King-sun Fu.

Paper received 7 May 1985

270

NDT INTERNATIONAL . AUGUST 1 9 8 6