Object pose determination using synthetic discriminant functions

Computers ind. Engng Vol. 21, Nos 1~,, pp. 349-353, 1991 Printed in Great Britain. All rights reserved

OBJECT

POSE

0360-8352/91 $3.00+ 0.00 Copyright © 1991 Pergamon Press plc

DETERMINATION USING FUNCTIONS

Sherri

SYNTHETIC

DISCRIMINANT

L. Messimer

Department of Industrial and Systems Engineering University of Alabama in Huntsville Huntsville, AL 35899

1.

INTRODUCTION

The increase in automated manufacturing and assembly operations has resulted in h e i g h t e n e d interest in robot vision systems for material handling tasks. The principal goal of robot vision is to increase the flexibility with which a robot can interact with its environment. It is desirable to provide the robot with the ability to recognize objects and to determine the position and orientation (or pose) of the object. This type of information is indispensable in an automated handling task such as preparing a robot gripper to grasp a component for insertion into a subassembly. This paper presents a method for determining whether a steel brake plate is in one of four possible orientations. This method utilizes Synthetic Discriminant Functions(SDFs), which are also known as linear combinatorial filters and m i n i m u m average correlation energy filters. Although these functions were originally used in optical pattern recognition systems, they are now also being u t i l i z e d in digital image processing systems [1],[4]. The development of the SDF will be discussed after a brief discussion of the brake plate m a n u f a c t u r i n g system. 2.

BRAKE

MANUFACTURING

The m a n u f a c t u r i n g of the brake plates begins with a stamping and deburring operation. They are then transported to an adhesive coating station via conveyor (Figure i) . Once the adhesive has been sprayed on the surface, the brakes are moved into an area where they are bonded with pads. The orientation of the plates is of importance in the adhesive coating operation as only the front of the parts should be coated with adhesive. Also, the top of the part should be forward in preparation for the subsequent bonding operation. The brake plates lie flat on a conveyor belt and are confined to a small area as depicted in Figure i. In order to ensure the correct orientation for both the adhesive coating operation and the subsequent application of the pad, a procedure for pose determination was required with the stipulation that the procedure take no more than three seconds to execute. If the pose is incorrect, the information is transmitted to a robot and appropriate action is taken. There are four possible orientations and action is required in three of these in order to rectify the situation:

i. 2. 3. 4.

Front Front Back Back

up, up, up, up,

i. 2. 3. 4.

top forward b o t t o m forward top forward b o t t o m forward

none (correct orientation) 180 degree yaw roll 180 degree roll rotation 180 degree roll and yaw rotation

The plates can be skewed within the rectangular area, as long as the front is up and the top is forward. There are presently sixteen different types of steel brake plates of various shapes and sizes. Only one type of brake plate is in production at any given time, so no part identification is necessary.

349

350

Proceedings of the 13th Annual Conference on Computers and Industrial Engineering

4" X 14" Conveyor Belt Area

Motion Detector

Directic

J

of Motio

I I

Figure

3.

SYNTHETIC

i. B r a k e P l a t e

DISCRIMINANT

Configuration

FUNCTION

(SDF)

DEVELOPMENT

Synthetic Discriminant Functions are composite images which are designed to act as a filter to locate an object regardless of its o r i e n t a t i o n [iJ, [2], [4]. A l t h o u g h SDFs are often u t i l i z e d to recognize an instance of an object in a c l u t t e r e d scene such as is r e q ui r e d in a missile g u i d a n c e system, they can also be used to determine if an object is in a correct orientation. In this p a r t i c u l a r research, a c o m p o s i t e image must be s t o r e d for each of the four d i s t i n c t o r i e n t a t i o n s . These composite images are c o n s t r u c t e d from the object in q u e s t i o n by p o s i n g the object in various r a n d o m positions. The images from each of these positions may be c o m b i n e d in various ways; summing linear combinations is one often used method [5]. Unfortunately, the manner in which these images are combined is often trial and error. Consider Figure 2 which illustrates three functions: Sum, bitwise AND, bitwise OR. All the images are combined with their centroids aligned. The SUM r e s u l t s from a d d i n g the i n t e n s i t y of each p i x e l in each image and d i v i d i n g by the number of images. This is an average image of the random orientations. The most o v e r l a p p e d areas tend to bright white saturation at the expense of the n o n - o v e r l a p p e d areas which appear darker. The AND results from the bitwise AND of each pixel in each image to the c o r r e s p o n d i n g pixels in the other images. Each image acts as a mask to all other images. Common o v e r l a p p e d areas are retained, n o n - o v e r l a p p e d areas are b l a c k e d out. The result is an image of the most common o v e r l a p p e d features. The OR results from the bitwise OR of each pixel in each image to the c o r r e s p o n d i n g pixels in the other images. Each image is bitwise ORed to all other images. The result is the c u m u l a t i v e sum of f e a t u r e s in e a c h o r i e n t a t i o n without intensity saturation. Other functions were also c o n s i d e r e d such as logical OR and logical AND.

Figure 2.Three Synthetic Discriminant Functions

Sum

Bitwise A N D

Bitwise OR

Messimer: Synthetic Discriminant Functions

351

O n c e t h e SDF h a s b e e n d e v e l o p e d a n d the c o m p o s i t e i m a g e s t o r e d in the database, it acts as a f i l t e r to a n e w input image. Again, the d e t e r m i n a t i o n of the a p p r o p r i a t e m a n n e r in w h i c h to u s e t h e f i l t e r is s o m e w h a t t r i a l and error. C o n s i d e r F i g u r e 3 w h i c h i l l u s t r a t e s the u s e of the AND SDF. Here the test i m a g e (original) is O R e d w i t h the SDF p r o d u c i n g a r e s u l t w h i c h is very s i m i l a r t o t h e o r i g i n a l image. T h e r e s u l t i n g i m a g e is s u b t r a c t e d f r o m the o r i g i n a l , as in a t e m p l a t e m a t c h i n g o p e r a t i o n [3]. F i g u r e 4 i l l u s t r a t e s the same A N D SDF n o w O R e d w i t h the i m a g e of the b r a k e p l a t e t u r n e d on its back. W h e n the o r i g i n a l r e v e r s e d i m a g e is O R e d w i t h the SDF the r e s u l t i n g o b j e c t is not like t h e o r i g i n a l , and thus would fail to be matched with original =empiate.

SDF F i g u r e 3. SDF: AND Comparison

SDF

Original Correctly Method:

Oriented

Result Plate

OR

Original

F i g u r e 4. Plate Reversed. SDF: AND C o m p a r i s o n Method: OR

Result

352

Proceedings of the 13th Annual Conference on Computers and Industrial

Figures 5 and 6 show similar an A N D c o m p a r i s o n m e t h o d .

results

SDF

OR

SDF

Original

Fiqure SDF:

for the

5.

Correctly

in c o n j u n c t i o n

with

Result Oriented

Plate.

OR

Comparison

SDF

Method:

AND

Original Figure SDF:

used

Engineering

6.

Plate

Result Reversed.

OR

Comparison

Method:

AND

A l t h o u g h d i f f e r e n t d i s c r i m i n a n t f u n c t i o n s c o u l d h a v e b e e n c h o s e n for e a c h of the f o u r p o s s i b l e orientations, initial results f r o m the O R SDF appeared to provide the greatest opportunity for discrimination. Four c o m p o s i t e i m a g e s w e r e d e t e r m i n e d for e a c h of the four p o s s i b l e o r i e n t a t i o n s . The p r o c e d u r e u s e d to d e v e l o p the c o m p o s i t e images is as follows:

Messimer: Synthetic Discriminant Functions

I. P l a c e b r a k e Digitize image.

plate

face

up,

and

2. Capture image at six additional i0 degree increments.

top

353

forward

locations

under

by rotating

the

camera.

the plate

in 5-

3. Align centroids of each image and bitwise OR every pixel value to the corresponding pixel values in each of the seven images - this is the SDF, or composite image. 4. Repeat

4.

this procedure

for each of the four positions.

Testing

Once the four composite images are stored in the database, they against input images to d e t e r m i n e if the o r i e n t a t i o n of the successfully determined. The procedure is as follows: i. Place brake plate as "Original"

under

the camera

in any position.

can be tested plate can be

Store this

image

2. Compare with SDF c o r r e s p o n d i n g to correct o r i e n t a t i o n using the AND comparison method. That is, bitwise AND the composite image to the captured image of the test brake plate. 3. Store this 4. Determine

image as "Result". if "Result"

is the same as "Original"

by template

matching.

5. D e t e r m i n e the correlation coefficient between the image "Result" and "Original". The c o r r e l a t i o n c o e f f i c i e n t is a m e a s u r e of the s i m i l a r i t y between two images. If the coefficient is above .90, the images are said to be equal. 6. If "Result" and "Original" composite image and begin testing. 7. Continue 5.

CONCLUSIONS

testing until AND

FUTURE

are not

correct

highly

orientation

correlated,

choose

another

is determined.

WORK

A l t h o u g h the above procedure resulted in an average "hit ratio" of 93%, the average time of execution doubled the stipulated time of three seconds. The largest n u m b e r of misses o c c u r r e d on parts which have almost the exact features on the top and b o t t o m of the part. Additional considerations needs to be given to the cut-off value of the correlation coefficient. Also, other methods such as Neural N e t w o r k s are also being c o n s i d e r e d in an effort to reduce the time required. 6.

REFERENCES

I.

P. Ceilo, Optical Techniques Press, London (1988).

for Industrial

2.

C.F. Hester and D. Casasent, Multivariant Technique for Multiclass Pattern Recognition, Applied Optics, 19, 1758-1761 (1980).

3.

M. James,

4.

A. Mahalanobis, B.V.K. Vijaya Kumar, and D. Casasent, Minimum Average Correlation Energy Filters, Applied Optics, 26, 3633-3640 (1987).

5.

S.R.F. Sims and J.A. Mills, Synthetic Discriminant Function (SDF) Filter Performance Evaluations, SPIE Vol. 1297 Hybrid Image and Signal Processing II, 110-121 (1990).

Pattern Recognition,

Wiley,

Inspection,

New York

Academic

(1988).