- Email: [email protected]
A transputer-based automated visual inspection system for electronic devices and PCBs
ELSEVIER
Optics ond Losers in Engineering 22 (19YS) 161-180 Copyright 0 1995 Elsevier Science Limited Printed in Northern Ireland. All rights reserved 014%X166/95/$9.50 0143~8166~94)ooo44-1
A Transputer-based Automated Visual Inspection System for Electronic Devices and PCBs
Earn Khwang
Teoh & Dinesh
P. Mital
School of Electrical and Electronics Engineering, Nanyang Technological Nanyang Avenue, Singapore 2263 Republic of Singapore
University,
ABSTRACT Computer vision has been gaining popularity for industrial applications, and the transputer offers a cost-effective vision hardware platform for an automated vision system In this paper, component level inspection was carried out on the integrated circuit (IQ diode, resistor and capacitor. Board level inspection was also implemented for inspection of multiple KS and missing components on assembled PCBs. The algorithms used were implemented for parallel processing using transputers. Through experiments, it was found that inspection of a component takes less than l-5 s, with an accuracy of above 85%. The transputer has also proven to out-perform sequential processing. The implementation of transputers for real-time image processing was successfully achieved.
1 INTRODUCTION The recent rapid growth in electronic industries has seen the production number of printed circuit boards (PCBs) increasing rapidly. To automate PCB production, many automated assembly machines have been developed and have proved their efficiencies. However, the inspection tasks, especially the visual inspection tasks, have not been effectively automated. Figure 1 shows the distribution and portion of the different tasks executed on the PCB production lines; 30% of the total tasks of the production lines involve inspecting the PCBs, and 60% of these tasks are visual inspection.’ A machine vision inspection system provides ways of cost-effectively increasing manufacturing visual inspection success and driving and sustaining desired process control capabilities. Until recently, cost-effective parallel processing was not available to 161
E. K. Teoh,
162
Fig. 1.
D. P. Mital
Distribution of tasks in a production line.
commerce and industry. The INMOS transputer avails new opportunities in performance, flexibility, and cost-effectiveness. Software can now be written to execute concurrently over a transputer network of arbitrary size, depending on the required performance. The computational demands of image processing have hence been met in a number of ways. The transputer provides a vision platform that is capable of continuous algorithm modification with existing hardware to accommodate various inspection requirements. Re-use of existing code and hardware family’s software compatibility can provide significant advantages when application development cost and speed performance are considered. Figure 2 presents a scenario whereby such a system can be flexible and also results in shorter development time.* The unique feature of this is that any number of transputers can be networked together to build multiprocessor systems. The performance is scaleable, giving a linear increase in performance as more processors are added to the system. Hence, any number of processes can be executed on a single transputer processor at the same time. For these reasons, parallel computers based on transputers are excellent computing tools for real-time processing because programs can be configured to run on the multiple transputers.3 Minimizing hardware size and complexity is an important goal in the design of real-time image processing. Transputer links provide an integrated solution to communication requirements within a computer. The paper attempts to exploit the advances in parallel processing in transputers to perform image processing applications. The algorithms
Transputer-based
automated
visual
inspection
163
system
Brilliant idea Solves high speed processing problems
Build Hardware and Software
Make It fast Make It inexpenshfe Make If programmable
Brilliant idea Solves a vlslon problem
r---i Program
on
Machine
I
--l--
L
development cycle
Fig. 2.
Hardware
concept-innovative.
for inspection of components at component and board levels are developed for implementation, using parallel processing for real-time application. Components inspected are the integrated circuit (IC), diode, resistor and capacitor. Inspection is carried out for type and/or orientation of the component.
2 A TRANSPUTER-BASED
VISUAL
INSPECTION
SYSTEM
2.1 System overview The system set up is shown in Fig. 3. The IBM PC/AT computer is used as the host for the transputer. The colour CCD converts the visual information into electrical signals (video signals) recognized by the
164
E. K. Teoh. D. P. Mital
EC MULTI SYN. MONITOF
Fig. 3.
System setup.
image grabber, and the image is displayed on the monitor. Front diffuse lighting is used for illumination. 2.2 Process flow The flowchart for the process of component inspection is shown in Fig. 4. In this process, a set of rules must be formulated to detect for irregularities in the components from a reference component. This set of rules is formulated by analysing features that can uniquely distinguish between components.4,s Description of these features is the central issue in the design of vision systems in the sense that these features affect not only the complexity of recognition algorithms, but also their performance. Hence, from these features, the correct component type and orientation can be determined. Auto-alignment and feature enhancement processing are needed to maintain consistency for these features throughout the inspection process. 2.3 Parallelism of the program Figure 5 shows a block diagram overview of the parallelism of the image processing task in transputers. The program consists of the ‘master’ and ‘worker’ tasks. The actual image is broken into subsets and packaged into appropriate ‘packets’ and sent to the worker for processing. The results coming back from the worker are then
Tanspurer-based
automated visual inspection system
AUTO
ALICHEHT
lbW%MLDIffi
::. ;... .i:. HISToCRAM BINARIUTI~
COLOUR TRANSFORMATION @ES-MIS0
INSFECTING PROCESS
0
LEA!4TmANsouuEOFORROR
REcoGNITIoN FJfwcEss
Fig. 4.
Main
.......
. . . . . . . . . . .._i
TElQLAlE
;
nATcHINs
SwoFDIFFERmEE
:...
Process flow of component
Thread
Send Thread
I Master
Fig. 5.
Task
Parallelism
inspection.
j Worker Task :,______ ____ ____ :1 i:c
for image processing.
’
165
166
E. K. Teoh. D. P. Mital
combined to form the feature representing the whole image. The worker is considered to be an independent processor performing a task independent of other workers and there is no communication between the workers. The transputer, operating in the processor farm configuration, has one master task, but can have many workers distributed throughout the transputer. The number of workers is limited by the number of transputers and their memories. 2.4 Dynamic windowing and auto-alignment These algorithms are implemented to cater for any shift in component during the teach and inspection stages for IC, diode and resistor. This is to overcome the effect of inconsistency of the features extracted from the component due to the translation of the image. The corner is detected and auto-alignment performed so that the component will always lie within the area of interest even allowing for shift in the component’s location during inspection. In cases where the component needs to detect for wrong orientation, diagonal corners detection may be necessary to centre the window or area of interset over the component. This process of auto-alignment consists of four stages. The flow for the dynamic windowing and auto-alignment is shown in Fig. 6.
Fig. 6.
Corner detection process.
Transputer-based automated visual inspection system
167
Binarizing the image will reduce the sensitivity of the inspection to variation in the lighting. Edge enhancement using the Roberts Cross Operator is then performed on the defined area to highlight the edge of the component. Row and column scanning are then performed individually. The row and column with the highest value are the coordinates of the corner for the component. The area of interest is then appended according to the corner detected. The corner is eventually identified as shown in the figure.
3 COMPONENT
LEVEL
INSPECTION
The features and algorithms used for inspection are deemed the most suitable amongst the features and algorithms tested. For each type of component to be inspected, both the technique used and program implementation of the inspection algorithms on the transputer system are described. 3.1 Types of component
inspected
3.1.1 IC
(i) Technique used. The features needed to discriminate between the different types of IC lie in the markings on the body. Figure 7 shows the segmentation of the area of interest on the IC. The area of interest is divided into 10 segments. The binarized pixel values within each segment are summed up and stored in segment arrays. These segment arrays of the pixel contents form the feature for the IC. Correct match for orientation and type is found when there is a forward match (i.e. comparing reference IC segment 1 with the inspected IC segment 1, etc.). For wrong orientation, it can be detected by comparing the value of the segments in the reverse order (i.e. reference IC segment 1 with the inspected IC segment 10, etc.). However, if there is no match, then the IC is judged to be of the wrong type. (ii) Program implementation. For this module, the worker task involves binarizing the subset of the image sent to it for processing and
168
E. K. Teoh, D. P. Mital
Segment
Label
1 1
2
3
4 ,:oms6t?
8
9
lo
Correct Type & Correct Orientation
123456759X
. Correct Type a Wrong Orientation
Wrong I.C.
Fig. 7.
IC inspection.
summing up these pixel values. The master task, on receiving the subsets of the result, will sort them accordingly into the various segments. The ‘teach’ and ‘inspection’ phases both involve these processes. In the inspection phase, after the segment feature is collected, the recognition process involves the checking of these features against the set stored in the teach phase. Segment 1 in the
Transputer-based automated visual inspection system
169
teach phase is compared with the segment 1 of the inspect phase and so on. If the differences between all these segments are within the certain tolerance set, then the inspected IC is considered of the same type and orientation as the IC used in the teach phase. Should any of these comparisons yield a difference greater than the set tolerance, then a second check is carried out. In the second check, the segments in the teach phase are compared with the segments of the inspect phase in the reversed order, i.e. the segment 1 of the teach phase is compared against the segment 10 of the inspect phase. If the differences of these comparisons is less than the set tolerance then the inspected IC is judged to be of the same type as, but of different orientation, the IC in the teach phase. Again, should the difference of these comparison be greater than the set tolerance, then the IC is judged to be the wrong IC.
3.1.2 Diode
(i) Technique
used. In the diode inspection,
the orientation or the polarity is to be identified. This module is catered for by black-bodied diodes. Similar to IC inspection, the diode image is also a binary image (black and white). The feature needed to discriminate between the direction or polarity lies in the polarity marking (white band). The technique used to identify the polarity of the diode is simpler since there is a distinct difference in the pixel values between the right and left halves of the diode. The idea is to sum up the binarized pixel values on each half and compare them with each other to determine the polarity. This feature is a generic property of the diode and by itself is enough to identify the polarity direction of the diode.
(ii) Program implementation. The worker task is again similar to the
worker task in the IC inspection, i.e. performing binarization of the subsets of the image sent to it for preprocessing and summing up the pixel values. The master task will then separate the results accordingly into the right and left half of the diode. A comparison of the pixel values of each half is done to determine the direction of the diode. This process is common to both the teach and inspect phases. In the recognition process, the direction of the diode under inspection is compared to the direction of the diode stored in the teach phase. If the directions are identical, then the diode is judged to be correct in polarity. Otherwise the diode is judged to be of wrong polarity.
170
E. K. Teoh, D. P. Mital
3.1.3 Resistor
(i) Technique used. Unlike the IC or diode where the essential features can be extracted by binarization, this approach is not applicable to resistor inspection because the essential feature of the resistor is its colour bands. Colour images contain more useful information as compared to grey-scale images.6 In the resistor inspection, the type of the resistor is compared against the referenced resistor. The identification of the resistor is independent of its orientation. An important characteristic of the standard resistor values is that there are rarely or never two values of the resistor that have their colour bands similar to each other. For example, there is a standard resistor value of 4.7 kQ which is represented by the colour bands yellow, violet and red, but there is no standard resistor value of 7.4 kR (violet, yellow and red). In other words, the set of colour bands in a standard resistor value is unique and no other standard resistor values have the same set. By making use of this characteristic in the standard resistor, it is only necessary to extract the colour bands signature of the reference resistor value and use it to discriminate against other resistor values.7 The colour of a pixel in a colour image is usually given as three values corresponding to the tristimulus values R (red), G (green) and B (blue). There are various other representations of colour information. One of the most popular methods is to describe the colour information using the HSI set of colour features-hue (H), saturation (S) and intensity (I).8 This is found to be more stable and less sensitive to slight variation in light intensity. Therefore, the technique used is to transform the RGB to HSI colour space where the hue information is used: z
1 JR+G+W
3 I =
z,,
’
z
2
p-w
z3 =(2xG)-R-B
2’
4
s =W*)* + v3>*19
H = arctan
: 02
The colour bands only occupy a small area of the whole resistor, thus enlarging the resistor by zooming in will only help to a certain extent. Since the interest is in the colour bands, the background and the body of the resistor (not the colour bands area) do not contribute any useful information; it is useful to eliminate them so that only the colour bands features are extracted. In processor farming, the image in the area of interest is divided into packages (subsets of the whole image) to be
Transputer-based automated visual inspection system ‘PACKRS’
I
I
I
I
II
‘PACKETS’
Fig. 8.
CONTAINING
II
ONE CoLouR
I
l\Ill/I
171
IN=ORNATION)
I
I
CONTAINIffi NOfE WAN ONE COLOW (COLOW7 BANX
I
I
FEATlRES)
Selective colour band feature extraction.
processed by the worker task. The percentage of a single colour level in the package can be used to distinguish whether the package contains useful (colour band features) or the irrelevant (background or the resistor body) information. Since the colour bands on the resistor are small, it is unlikely that the package can lie entirely within a single colour band. When the package shows a high percentage of the pixels having a single colour level, then the probability is that this package is a subset of the background or resistor body and thus can be discarded. This is the technique used for selective colour band feature extraction. Figure 8 shows an example of the ‘packets’ or subsets of the image which are selectively used to extract the colour bands’ features. It can be seen that besides the colour bands, the areas between the body and the background are also regarded as useful because there is more than one colour in these ‘packets’. Though the selective technique above is used to enhance the relevant colour bands, irrelevant information may still dominate the hue information. Hence, in the recognition process, the irrelevant background or resistor body colour is suppressed before the matching process (i.e. by disregarding the background information). For the histogram of hue levels built from the selective colour band extraction algorithm, the matching algorithm used is the square root of the sum of the difference between the colour bands’ features extracted in the teach and inspect modes.
172
E. K. Teoh, 0. P. Mital
(ii) Program
implementation.
The worker task in this module involves the colour transformation from RGB to HSI. The transformed colour (hue) information is then sorted into the histogram of hue levels. When the package is processed, a check is done to determine if there is a high concentration of above 220 out of 240 pixels in the package in one hue level, then the package is considered to be containing only redundant information. Subsequently, the result of this worker package will be nulled before returning to the master task. The teach and inspect phases both involve the same process of extracting the two sets of histograms of hue level from the resistor image. In the recognition process, the matching algorithms will use the two sets of histograms from the teach and inspect phases to compute that if the difference between each of the matching algorithms is less than a set value then the flag is set. Only when both of these flags are set is the resistor under inspection considered to be matched with the reference resistor in the teach phase. Otherwise the resistor is considered to be wrong.
3.2 On-line testing results and discussion
The inspection results obtained shown in Table 1.
for inspecting
each component
are as
3.2.1 IC From the results of IC inspection, it was observed that, when the IC is of the correct type and orientation, the difference produced from the comparison is small as compared to the IC being of the correct type, but wrong orientation or when the wrong IC is inspected. In the case when the IC inspected is the correct type, but wrong orientation, when
Components Component type
TABLE 1 Level Inspection
Accuracy W)
Integrated circuit Diode Resistor
87 c. 100 87
Results
inspection time
No. of components
(s)
W’)
0.94 0.86 0.94
3800 4180 3830
Transputer-based automated visual inspection system
173
the segments in the teach phase are compared with the segments in the inspect phase in the reversed order the results again show a small difference (see Fig. 7). The overall accuracy of the IC inspection module is 87%. The wrong identification of the IC is due mainly to two reasons: the shadow cast around the IC and the lighting condition. These reasons may result in the inaccuracy of the dynamic windowing and auto-alignment routine, and the proper segmentation of the IC is dependent on the correct positioning of the window over the IC. For these reasons, emphasis is placed on proper calibration prior to inspection.
3.2.2 Diode From the results of diode inspection, it is observed that the sum of the binarized pixel value of the right and left segments of the diode showed a clear difference. This makes the identification of diode polarity easy and reliable. Using this simple approach to determine the orientation of the diode polarity yielded a high degree of accuracy close to 100%. Wrong results occur occasionally due mainly to shadow and the lighting condition around the diode.
3.2.3 Resistor For resistor inspection, it was found that the technique of transforming the RGB to HSI colour spaces proposed by Ohta et al. in Ref. 6 was better in its ability to provide a better discriminating colour information. This is because it provides better spread in the hue range for the primary colours. It was also found that inspection of the resistor is consistent regardless of the orientation of the resistor. The resistor inspection module has shown an overall accuracy of 87%. Wrong identification of the resistors is due mainly to the glare coming from the reflection of light from the body of the resistor. A circular polarized filter is used to reduce the glare to some extent but is unable to cut it out completely. This glare will interfere with the colour bands’ features. Another factor causing false inspection is due to the uneven colour bands’ thickness that sometimes may occur. False inspection results may also occur if there are copper tracks running on the PCB in the area around the resistor. This is because the change of colour due to these copper tracks will be included in the colour bands features, hence reducing the reliability of the module. This effect can be
174
E. K. Teoh, D. P. Mital
reduced or cancelled out if the area of interest can be further restricted to within the body of the resistor itself.
4 BOARD
LEVEL
INSPECTION
Building on the component level inspection program, board level inspection can look at multiple components at one time. The algorithms used for inspection are based on those tested in the earlier inspection program. 4.1 Missing component
inspection
4.1.1 Technique used In industrial applications where a vision system is used in the inspection of PCBs, missing component inspection is a prerequisite initial process after component placement. This is to ensure that all the components are present before the board proceeds to the next stage. This filtering will reduce the time and complexity of the functional testing at subsequent stages. The components inspected are the resistor, capacitor, diode and IC. To detect for missing components, essentially few features are needed, If the component is missing, the simplest feature will show large variation. Therefore, for missing component inspection, the feature used is the cumulated binarized value of the individual component. 4.1.2 Program implementation This module is again divided into two phases, i.e. the teach phase and inspection phase. During the teach phase, the user is required to identify the location of the component, which involves marking of the diagonal coordinates of the component. Once the coordinates are marked, the program will automatically adjust the windows size, such that it is made up of an integer number of packets size. The new adjusted coordinates and the feature in the defined area (component) is then computed and stored in the respective component arrays. A facility to delete the component from being inspected is also implemented. Once the components are marked and ready for inspection, the total number of packets for all the components are computed. This is necessary as the program needs to monitor the number of packets required to send out to the worker for processing. The inspection
Transputer-based automated visual inspection system
175
phase will inspect for missing components on the board under test. In this phase, the program will compute for the features of the individual component at the positions previously taught during the teach phase. This result is then compared with features stored in the component arrays to detect for missing components. If the feature for an inspected component varies by more than a set threshold value, then the component is judged to be missing. 4.2 Multiple IC inspection 4.2.1 Technique used
In this module, the ICs are checked for type and orientation as well as the presence of the component. The technique for feature extraction in this module is similar to that used in a single IC inspection module. To inspect multiple ICs at the same time, the segment size has to be reduced. The same algorithm can be used to discriminate between the type and orientation of the ICs. 4.2.2 Program implementation This module contains both the teach phase and the inspection phase. During the teach phase, adding the IC for inspection is done by clicking the top left corner of the IC to be inspected, using the mouse. These coordinates will be stored and used for subsequent inspection. Corner detection will be performed on that edge. As this component needs to be inspected for orientation, centring the window on the component is very important. This is achieved by performing an additional adjacent corner detection. Based on these two corners’ coordinates, the program will compute the exact location to position the window. When the required components have been selected, the program will compute the reference features for each of the selected IC. In the inspection phase, the program will refer to the teach phase to obtain the respective coordinates of each of the ICs so that the necessary adjacent corner detection and centring of the windows is possible. In cases where the component is not present, the corner detection for that component will fail and the component status flag for the component missing will be set. Upon completion of the corners’ identification, the actual computation begins. The matching algorithm will be performed on each component at a time. For a particular component, a maximum of two sequences are used in the matching process. If the direct sequence (for example, segment 1 of component 1 of the teach phase is compared with the same segment of component 1 of the inspect phase) satisfies the criteria, then the IC is considered to
176
E. K. Teoh, D. P. Mital TABLE
2
Board Level Inspection Results Component type
Accuracy W)
Missing component Multiple IC
Inspection timelcomp.
No. of components
6)
(h-7
85 85
be of the correct type and orientation. comparison is applied.
0.01542 0.1
233 463 36 000
If it fails, a reverse sequence
4.3 On-line testing results and discussion The inspection results obtained for inspection of each component are shown in Table 2. For the inspection of missing components, the accuracy shown in the experiments conducted is 85%. The reasons for the wrong identification in this module are similar to those discussed in Section 3.2. As for the inspection of multiple IC, the accuracy obtained is 85%, which is slightly poorer than that in the IC inspection module. This is because some of the features in the IC are lost when the IC image has to be reduced so that more ICs can be inspected at one time. Also, as the camera has to cover a broader area, the lighting has to be even throughout the area under inspection. This is to ensure that the two corners, dynamic windowing and auto-alignment, can work properly for all the ICs to be inspected. Further evaluation is performed to compare between sequential and parallel processing for missing component inspection. It is observed that, for parallel processing, the time taken for inspecting a few components is more than sequential; however, as the number of components to be inspected increases, parallel processing has the advantage. It is also observed that parallel processing shows a small linear increase in time as compared to sequential, which exhibits an exponential increase in timing as the number of components increases. The reason for the above observations is because when fewer components are inspected, the communication overheads in parallel processing are more than in sequential processing. However, when the number of tasks increases, the time taken by the sequential processing increases rapidly. This is because with more tasks, there will be more data and pointers swapping, more tasks to keep track of, etc. Hence,
Transputer-based
automated visual inspection system
Number Fig. 9.
A comparison
of
177
Components
of sequential
versus parallel processing.
the bottleneck in the sequential processing is the speed of the processor. The bottleneck of sequential processing does not exist in the processor farming parallelism. Each worker itself is a processor and the number of workers that can exist in the transputer is limited only by the number of transputers and their memory size. Hence, it can be concluded that the parallel approach is more beneficial as the number of components to be inspected increases. Figure 9 shows the comparison between sequential and parallel processing. In the multiple IC inspection, since the number of pixels occupied by each IC is less, the processing time needed is also less. The accuracy demonstrated is slightly poorer than that of the IC inspection module. This is because some of the features in the IC are lost when the IC image has to be reduced so that more ICs can be inspected at one time.
5 IMPLEMENTATION ASPECTS OF AN INDUSTRIAL TRANSPUTER-BASED VISION SYSTEM The transputer offers a powerful, flexible and cost-effective computer vision hardware for an industrial automated vision system. It is of relatively low cost as compared with other vision system hardware. Its flexibility in its internal architecture enables the transputer to choose the best configuration for any application. Overcoming the bottlenecks in terms of processing capacity and speed can be achieved by simply increasing the number of transputers.
E. K. Teoh, D. P. Mital
178
1
.‘.
-.
MASTER COMPUTER
.VICFoUR
\ .\\
~--cmPlmeus
‘\
Fig. 10.
A transducer-based
automated
visual inspection
I
system.
Figure 10 shows a transputer-based automated visual inspection system set up on an assembly line. Inspection stations are located at two points in the assembly process. The first point is after the automatic component insertion and/or the SMT mounting. Two inspection stations are needed at this point. The first station is to inspect for missing components while the second station inspects for the component type and orientation as well as for improper insertion. The components involved include the IC, diode, resistor and capacitor. The third station for the inspection system is placed after the soldering stage to inspect the solder joint quality. The transputers used in the inspection task use the processor farming method. This is because the image analysis involves tasks that vary in complexity for different components. Hence, the processor farming method is suitable. A host computer is needed to interface with the transputers used for the inspection system, together with the frame grabbers; hence, a computer is needed at each inspection station. These computers are then linked to a master computer to oversee and control the performance of the whole system. To facilitate the inspection of a large PCB, an X-Y table is
Transputer-based
visual inspection
179
recommended for all inspection stations. A two camera set-up is needed for inspection of components as well as improper insertion. The two cameras can be connected to two fransputer-based frame grabbers and connected to the transputers for image capture and analysis. In the solder inspection station, the camera and lighting are placed below the conveyor belt. The inspection area will allow the camera to have an unrestricted view of the solder area; hence, the X-Y table at this stage will have its bottom hollowed out for the camera to view the solder area. Front diffused lighting is used for all the inspection stations-since the lighting over the entire PCB should be as even as possible, this type of lighting is best suited. To overcome bottlenecks in the inspection process, it is proposed that more than one system be set up for the second and third inspection stations as the work involved at these two stations is more intensive. Two stations are shown in Fig. 10. The assembled PCB can be inspected by either station, which are laid out parallel to one another. There is a transition table that will handle and distribute the flow of the assembled PCB for inspection. To improve further on the processing speed of this stage, the integration of a digital signal processor (DSP) into the system can be considered. Preprocessing of the images involved at this stage will improve the reliability of the system. Since most preprocessing (noise removal, etc.) involves a large volume of data to be processed, such a task is best suited for a DSP. Also, some of the algorithms used for inspection will inevitably involve number crunching as well. The DSP can also be used to improve on the processing speed of these algorithms. Hence, low level processing (for example, edge detection) can be handled by the DSP while the transputer can be used to handle medium (for example, edge linking, segmentation, description and recognition) and high level processing (for example, applying artificia1 intelligence techniques), tasks to which a transputer is better suited.
6 CONCLUSION With the increasing popularity of using computer vision to replace manual inspection in industry, the transputer offers a promising option for a cost-effective vision system. In this paper, parallel processing using transputers to achieve real-time image processing for component inspection was explored. Inspection of components at component and board levels was developed using various algorithms. Each algorithm was tailored for each component to achieve accuracy and reliability.
180
E. K. Teoh, D. P. M&al
From the experimental results, inspection of components at a level above 2ooO units h-l has been achieved with high accuracy. Parallel processing in the transputer has also demonstrated its advantage over sequential processing. Hence, with the flexibility in transputer architecture, the transputer does possess the potential for further improvement in real-time applications.
ACKNOWLEDGEMENTS The authors wish to extend their gratitude and appreciation to Professor Brian Lee, Dean of Electrical and Electronic Engineering, Nanyang Technological University, for all his initiatives and support for this research project.
REFERENCES 1. Hata, S., Hagimae, K., Hibi, S. & Gunji, T., Assembled PCB visual inspection machine using image processor with DSP. In Proc. ZECON, pp. 572-7. 2. LeBeau, C. J., Machine vision in semiconductor manufacturing. Advance Imaging Magazine (May) (1991) 54-8. 3. Harp, G., Transputer Applications, 1st edn, Pitman, London, 1989. 4. Schalkoff, R. J., Digital Image Processing and Computer Vision, 1st edn, John Wiley & Sons, New York, 1989. 5. Gonzalez, R. C. & Win&, P., Digital Image Processing, 2nd edn, AddisonWesley, Canada, 1987. 6. Berry, D. T., Colour Recognition using Spectral Signatures, Heriot-Watt University, Edinburgh, UK, 1987, pp. 31-5. 7. Yong, I. N., A robotic vision system with incorporation of artificial intelligence for industrial application, Master Thesis, Nanyang Technological Institute, Singapore, 1988. 8. Kurz, B. J., Optimal quantization for colour displays. IEEE Trans. Pattern Anal. and Machine Intell. (1983) 217-24.
Optics ond Losers in Engineering 22 (19YS) 161-180 Copyright 0 1995 Elsevier Science Limited Printed in Northern Ireland. All rights reserved 014%X166/95/$9.50 0143~8166~94)ooo44-1
A Transputer-based Automated Visual Inspection System for Electronic Devices and PCBs
Earn Khwang
Teoh & Dinesh
P. Mital
School of Electrical and Electronics Engineering, Nanyang Technological Nanyang Avenue, Singapore 2263 Republic of Singapore
University,
ABSTRACT Computer vision has been gaining popularity for industrial applications, and the transputer offers a cost-effective vision hardware platform for an automated vision system In this paper, component level inspection was carried out on the integrated circuit (IQ diode, resistor and capacitor. Board level inspection was also implemented for inspection of multiple KS and missing components on assembled PCBs. The algorithms used were implemented for parallel processing using transputers. Through experiments, it was found that inspection of a component takes less than l-5 s, with an accuracy of above 85%. The transputer has also proven to out-perform sequential processing. The implementation of transputers for real-time image processing was successfully achieved.
1 INTRODUCTION The recent rapid growth in electronic industries has seen the production number of printed circuit boards (PCBs) increasing rapidly. To automate PCB production, many automated assembly machines have been developed and have proved their efficiencies. However, the inspection tasks, especially the visual inspection tasks, have not been effectively automated. Figure 1 shows the distribution and portion of the different tasks executed on the PCB production lines; 30% of the total tasks of the production lines involve inspecting the PCBs, and 60% of these tasks are visual inspection.’ A machine vision inspection system provides ways of cost-effectively increasing manufacturing visual inspection success and driving and sustaining desired process control capabilities. Until recently, cost-effective parallel processing was not available to 161
E. K. Teoh,
162
Fig. 1.
D. P. Mital
Distribution of tasks in a production line.
commerce and industry. The INMOS transputer avails new opportunities in performance, flexibility, and cost-effectiveness. Software can now be written to execute concurrently over a transputer network of arbitrary size, depending on the required performance. The computational demands of image processing have hence been met in a number of ways. The transputer provides a vision platform that is capable of continuous algorithm modification with existing hardware to accommodate various inspection requirements. Re-use of existing code and hardware family’s software compatibility can provide significant advantages when application development cost and speed performance are considered. Figure 2 presents a scenario whereby such a system can be flexible and also results in shorter development time.* The unique feature of this is that any number of transputers can be networked together to build multiprocessor systems. The performance is scaleable, giving a linear increase in performance as more processors are added to the system. Hence, any number of processes can be executed on a single transputer processor at the same time. For these reasons, parallel computers based on transputers are excellent computing tools for real-time processing because programs can be configured to run on the multiple transputers.3 Minimizing hardware size and complexity is an important goal in the design of real-time image processing. Transputer links provide an integrated solution to communication requirements within a computer. The paper attempts to exploit the advances in parallel processing in transputers to perform image processing applications. The algorithms
Transputer-based
automated
visual
inspection
163
system
Brilliant idea Solves high speed processing problems
Build Hardware and Software
Make It fast Make It inexpenshfe Make If programmable
Brilliant idea Solves a vlslon problem
r---i Program
on
Machine
I
--l--
L
development cycle
Fig. 2.
Hardware
concept-innovative.
for inspection of components at component and board levels are developed for implementation, using parallel processing for real-time application. Components inspected are the integrated circuit (IC), diode, resistor and capacitor. Inspection is carried out for type and/or orientation of the component.
2 A TRANSPUTER-BASED
VISUAL
INSPECTION
SYSTEM
2.1 System overview The system set up is shown in Fig. 3. The IBM PC/AT computer is used as the host for the transputer. The colour CCD converts the visual information into electrical signals (video signals) recognized by the
164
E. K. Teoh. D. P. Mital
EC MULTI SYN. MONITOF
Fig. 3.
System setup.
image grabber, and the image is displayed on the monitor. Front diffuse lighting is used for illumination. 2.2 Process flow The flowchart for the process of component inspection is shown in Fig. 4. In this process, a set of rules must be formulated to detect for irregularities in the components from a reference component. This set of rules is formulated by analysing features that can uniquely distinguish between components.4,s Description of these features is the central issue in the design of vision systems in the sense that these features affect not only the complexity of recognition algorithms, but also their performance. Hence, from these features, the correct component type and orientation can be determined. Auto-alignment and feature enhancement processing are needed to maintain consistency for these features throughout the inspection process. 2.3 Parallelism of the program Figure 5 shows a block diagram overview of the parallelism of the image processing task in transputers. The program consists of the ‘master’ and ‘worker’ tasks. The actual image is broken into subsets and packaged into appropriate ‘packets’ and sent to the worker for processing. The results coming back from the worker are then
Tanspurer-based
automated visual inspection system
AUTO
ALICHEHT
lbW%MLDIffi
::. ;... .i:. HISToCRAM BINARIUTI~
COLOUR TRANSFORMATION @ES-MIS0
INSFECTING PROCESS
0
LEA!4TmANsouuEOFORROR
REcoGNITIoN FJfwcEss
Fig. 4.
Main
.......
. . . . . . . . . . .._i
TElQLAlE
;
nATcHINs
SwoFDIFFERmEE
:...
Process flow of component
Thread
Send Thread
I Master
Fig. 5.
Task
Parallelism
inspection.
j Worker Task :,______ ____ ____ :1 i:c
for image processing.
’
165
166
E. K. Teoh. D. P. Mital
combined to form the feature representing the whole image. The worker is considered to be an independent processor performing a task independent of other workers and there is no communication between the workers. The transputer, operating in the processor farm configuration, has one master task, but can have many workers distributed throughout the transputer. The number of workers is limited by the number of transputers and their memories. 2.4 Dynamic windowing and auto-alignment These algorithms are implemented to cater for any shift in component during the teach and inspection stages for IC, diode and resistor. This is to overcome the effect of inconsistency of the features extracted from the component due to the translation of the image. The corner is detected and auto-alignment performed so that the component will always lie within the area of interest even allowing for shift in the component’s location during inspection. In cases where the component needs to detect for wrong orientation, diagonal corners detection may be necessary to centre the window or area of interset over the component. This process of auto-alignment consists of four stages. The flow for the dynamic windowing and auto-alignment is shown in Fig. 6.
Fig. 6.
Corner detection process.
Transputer-based automated visual inspection system
167
Binarizing the image will reduce the sensitivity of the inspection to variation in the lighting. Edge enhancement using the Roberts Cross Operator is then performed on the defined area to highlight the edge of the component. Row and column scanning are then performed individually. The row and column with the highest value are the coordinates of the corner for the component. The area of interest is then appended according to the corner detected. The corner is eventually identified as shown in the figure.
3 COMPONENT
LEVEL
INSPECTION
The features and algorithms used for inspection are deemed the most suitable amongst the features and algorithms tested. For each type of component to be inspected, both the technique used and program implementation of the inspection algorithms on the transputer system are described. 3.1 Types of component
inspected
3.1.1 IC
(i) Technique used. The features needed to discriminate between the different types of IC lie in the markings on the body. Figure 7 shows the segmentation of the area of interest on the IC. The area of interest is divided into 10 segments. The binarized pixel values within each segment are summed up and stored in segment arrays. These segment arrays of the pixel contents form the feature for the IC. Correct match for orientation and type is found when there is a forward match (i.e. comparing reference IC segment 1 with the inspected IC segment 1, etc.). For wrong orientation, it can be detected by comparing the value of the segments in the reverse order (i.e. reference IC segment 1 with the inspected IC segment 10, etc.). However, if there is no match, then the IC is judged to be of the wrong type. (ii) Program implementation. For this module, the worker task involves binarizing the subset of the image sent to it for processing and
168
E. K. Teoh, D. P. Mital
Segment
Label
1 1
2
3
4 ,:oms6t?
8
9
lo
Correct Type & Correct Orientation
123456759X
. Correct Type a Wrong Orientation
Wrong I.C.
Fig. 7.
IC inspection.
summing up these pixel values. The master task, on receiving the subsets of the result, will sort them accordingly into the various segments. The ‘teach’ and ‘inspection’ phases both involve these processes. In the inspection phase, after the segment feature is collected, the recognition process involves the checking of these features against the set stored in the teach phase. Segment 1 in the
Transputer-based automated visual inspection system
169
teach phase is compared with the segment 1 of the inspect phase and so on. If the differences between all these segments are within the certain tolerance set, then the inspected IC is considered of the same type and orientation as the IC used in the teach phase. Should any of these comparisons yield a difference greater than the set tolerance, then a second check is carried out. In the second check, the segments in the teach phase are compared with the segments of the inspect phase in the reversed order, i.e. the segment 1 of the teach phase is compared against the segment 10 of the inspect phase. If the differences of these comparisons is less than the set tolerance then the inspected IC is judged to be of the same type as, but of different orientation, the IC in the teach phase. Again, should the difference of these comparison be greater than the set tolerance, then the IC is judged to be the wrong IC.
3.1.2 Diode
(i) Technique
used. In the diode inspection,
the orientation or the polarity is to be identified. This module is catered for by black-bodied diodes. Similar to IC inspection, the diode image is also a binary image (black and white). The feature needed to discriminate between the direction or polarity lies in the polarity marking (white band). The technique used to identify the polarity of the diode is simpler since there is a distinct difference in the pixel values between the right and left halves of the diode. The idea is to sum up the binarized pixel values on each half and compare them with each other to determine the polarity. This feature is a generic property of the diode and by itself is enough to identify the polarity direction of the diode.
(ii) Program implementation. The worker task is again similar to the
worker task in the IC inspection, i.e. performing binarization of the subsets of the image sent to it for preprocessing and summing up the pixel values. The master task will then separate the results accordingly into the right and left half of the diode. A comparison of the pixel values of each half is done to determine the direction of the diode. This process is common to both the teach and inspect phases. In the recognition process, the direction of the diode under inspection is compared to the direction of the diode stored in the teach phase. If the directions are identical, then the diode is judged to be correct in polarity. Otherwise the diode is judged to be of wrong polarity.
170
E. K. Teoh, D. P. Mital
3.1.3 Resistor
(i) Technique used. Unlike the IC or diode where the essential features can be extracted by binarization, this approach is not applicable to resistor inspection because the essential feature of the resistor is its colour bands. Colour images contain more useful information as compared to grey-scale images.6 In the resistor inspection, the type of the resistor is compared against the referenced resistor. The identification of the resistor is independent of its orientation. An important characteristic of the standard resistor values is that there are rarely or never two values of the resistor that have their colour bands similar to each other. For example, there is a standard resistor value of 4.7 kQ which is represented by the colour bands yellow, violet and red, but there is no standard resistor value of 7.4 kR (violet, yellow and red). In other words, the set of colour bands in a standard resistor value is unique and no other standard resistor values have the same set. By making use of this characteristic in the standard resistor, it is only necessary to extract the colour bands signature of the reference resistor value and use it to discriminate against other resistor values.7 The colour of a pixel in a colour image is usually given as three values corresponding to the tristimulus values R (red), G (green) and B (blue). There are various other representations of colour information. One of the most popular methods is to describe the colour information using the HSI set of colour features-hue (H), saturation (S) and intensity (I).8 This is found to be more stable and less sensitive to slight variation in light intensity. Therefore, the technique used is to transform the RGB to HSI colour space where the hue information is used: z
1 JR+G+W
3 I =
z,,
’
z
2
p-w
z3 =(2xG)-R-B
2’
4
s =W*)* + v3>*19
H = arctan
: 02
The colour bands only occupy a small area of the whole resistor, thus enlarging the resistor by zooming in will only help to a certain extent. Since the interest is in the colour bands, the background and the body of the resistor (not the colour bands area) do not contribute any useful information; it is useful to eliminate them so that only the colour bands features are extracted. In processor farming, the image in the area of interest is divided into packages (subsets of the whole image) to be
Transputer-based automated visual inspection system ‘PACKRS’
I
I
I
I
II
‘PACKETS’
Fig. 8.
CONTAINING
II
ONE CoLouR
I
l\Ill/I
171
IN=ORNATION)
I
I
CONTAINIffi NOfE WAN ONE COLOW (COLOW7 BANX
I
I
FEATlRES)
Selective colour band feature extraction.
processed by the worker task. The percentage of a single colour level in the package can be used to distinguish whether the package contains useful (colour band features) or the irrelevant (background or the resistor body) information. Since the colour bands on the resistor are small, it is unlikely that the package can lie entirely within a single colour band. When the package shows a high percentage of the pixels having a single colour level, then the probability is that this package is a subset of the background or resistor body and thus can be discarded. This is the technique used for selective colour band feature extraction. Figure 8 shows an example of the ‘packets’ or subsets of the image which are selectively used to extract the colour bands’ features. It can be seen that besides the colour bands, the areas between the body and the background are also regarded as useful because there is more than one colour in these ‘packets’. Though the selective technique above is used to enhance the relevant colour bands, irrelevant information may still dominate the hue information. Hence, in the recognition process, the irrelevant background or resistor body colour is suppressed before the matching process (i.e. by disregarding the background information). For the histogram of hue levels built from the selective colour band extraction algorithm, the matching algorithm used is the square root of the sum of the difference between the colour bands’ features extracted in the teach and inspect modes.
172
E. K. Teoh, 0. P. Mital
(ii) Program
implementation.
The worker task in this module involves the colour transformation from RGB to HSI. The transformed colour (hue) information is then sorted into the histogram of hue levels. When the package is processed, a check is done to determine if there is a high concentration of above 220 out of 240 pixels in the package in one hue level, then the package is considered to be containing only redundant information. Subsequently, the result of this worker package will be nulled before returning to the master task. The teach and inspect phases both involve the same process of extracting the two sets of histograms of hue level from the resistor image. In the recognition process, the matching algorithms will use the two sets of histograms from the teach and inspect phases to compute that if the difference between each of the matching algorithms is less than a set value then the flag is set. Only when both of these flags are set is the resistor under inspection considered to be matched with the reference resistor in the teach phase. Otherwise the resistor is considered to be wrong.
3.2 On-line testing results and discussion
The inspection results obtained shown in Table 1.
for inspecting
each component
are as
3.2.1 IC From the results of IC inspection, it was observed that, when the IC is of the correct type and orientation, the difference produced from the comparison is small as compared to the IC being of the correct type, but wrong orientation or when the wrong IC is inspected. In the case when the IC inspected is the correct type, but wrong orientation, when
Components Component type
TABLE 1 Level Inspection
Accuracy W)
Integrated circuit Diode Resistor
87 c. 100 87
Results
inspection time
No. of components
(s)
W’)
0.94 0.86 0.94
3800 4180 3830
Transputer-based automated visual inspection system
173
the segments in the teach phase are compared with the segments in the inspect phase in the reversed order the results again show a small difference (see Fig. 7). The overall accuracy of the IC inspection module is 87%. The wrong identification of the IC is due mainly to two reasons: the shadow cast around the IC and the lighting condition. These reasons may result in the inaccuracy of the dynamic windowing and auto-alignment routine, and the proper segmentation of the IC is dependent on the correct positioning of the window over the IC. For these reasons, emphasis is placed on proper calibration prior to inspection.
3.2.2 Diode From the results of diode inspection, it is observed that the sum of the binarized pixel value of the right and left segments of the diode showed a clear difference. This makes the identification of diode polarity easy and reliable. Using this simple approach to determine the orientation of the diode polarity yielded a high degree of accuracy close to 100%. Wrong results occur occasionally due mainly to shadow and the lighting condition around the diode.
3.2.3 Resistor For resistor inspection, it was found that the technique of transforming the RGB to HSI colour spaces proposed by Ohta et al. in Ref. 6 was better in its ability to provide a better discriminating colour information. This is because it provides better spread in the hue range for the primary colours. It was also found that inspection of the resistor is consistent regardless of the orientation of the resistor. The resistor inspection module has shown an overall accuracy of 87%. Wrong identification of the resistors is due mainly to the glare coming from the reflection of light from the body of the resistor. A circular polarized filter is used to reduce the glare to some extent but is unable to cut it out completely. This glare will interfere with the colour bands’ features. Another factor causing false inspection is due to the uneven colour bands’ thickness that sometimes may occur. False inspection results may also occur if there are copper tracks running on the PCB in the area around the resistor. This is because the change of colour due to these copper tracks will be included in the colour bands features, hence reducing the reliability of the module. This effect can be
174
E. K. Teoh, D. P. Mital
reduced or cancelled out if the area of interest can be further restricted to within the body of the resistor itself.
4 BOARD
LEVEL
INSPECTION
Building on the component level inspection program, board level inspection can look at multiple components at one time. The algorithms used for inspection are based on those tested in the earlier inspection program. 4.1 Missing component
inspection
4.1.1 Technique used In industrial applications where a vision system is used in the inspection of PCBs, missing component inspection is a prerequisite initial process after component placement. This is to ensure that all the components are present before the board proceeds to the next stage. This filtering will reduce the time and complexity of the functional testing at subsequent stages. The components inspected are the resistor, capacitor, diode and IC. To detect for missing components, essentially few features are needed, If the component is missing, the simplest feature will show large variation. Therefore, for missing component inspection, the feature used is the cumulated binarized value of the individual component. 4.1.2 Program implementation This module is again divided into two phases, i.e. the teach phase and inspection phase. During the teach phase, the user is required to identify the location of the component, which involves marking of the diagonal coordinates of the component. Once the coordinates are marked, the program will automatically adjust the windows size, such that it is made up of an integer number of packets size. The new adjusted coordinates and the feature in the defined area (component) is then computed and stored in the respective component arrays. A facility to delete the component from being inspected is also implemented. Once the components are marked and ready for inspection, the total number of packets for all the components are computed. This is necessary as the program needs to monitor the number of packets required to send out to the worker for processing. The inspection
Transputer-based automated visual inspection system
175
phase will inspect for missing components on the board under test. In this phase, the program will compute for the features of the individual component at the positions previously taught during the teach phase. This result is then compared with features stored in the component arrays to detect for missing components. If the feature for an inspected component varies by more than a set threshold value, then the component is judged to be missing. 4.2 Multiple IC inspection 4.2.1 Technique used
In this module, the ICs are checked for type and orientation as well as the presence of the component. The technique for feature extraction in this module is similar to that used in a single IC inspection module. To inspect multiple ICs at the same time, the segment size has to be reduced. The same algorithm can be used to discriminate between the type and orientation of the ICs. 4.2.2 Program implementation This module contains both the teach phase and the inspection phase. During the teach phase, adding the IC for inspection is done by clicking the top left corner of the IC to be inspected, using the mouse. These coordinates will be stored and used for subsequent inspection. Corner detection will be performed on that edge. As this component needs to be inspected for orientation, centring the window on the component is very important. This is achieved by performing an additional adjacent corner detection. Based on these two corners’ coordinates, the program will compute the exact location to position the window. When the required components have been selected, the program will compute the reference features for each of the selected IC. In the inspection phase, the program will refer to the teach phase to obtain the respective coordinates of each of the ICs so that the necessary adjacent corner detection and centring of the windows is possible. In cases where the component is not present, the corner detection for that component will fail and the component status flag for the component missing will be set. Upon completion of the corners’ identification, the actual computation begins. The matching algorithm will be performed on each component at a time. For a particular component, a maximum of two sequences are used in the matching process. If the direct sequence (for example, segment 1 of component 1 of the teach phase is compared with the same segment of component 1 of the inspect phase) satisfies the criteria, then the IC is considered to
176
E. K. Teoh, D. P. Mital TABLE
2
Board Level Inspection Results Component type
Accuracy W)
Missing component Multiple IC
Inspection timelcomp.
No. of components
6)
(h-7
85 85
be of the correct type and orientation. comparison is applied.
0.01542 0.1
233 463 36 000
If it fails, a reverse sequence
4.3 On-line testing results and discussion The inspection results obtained for inspection of each component are shown in Table 2. For the inspection of missing components, the accuracy shown in the experiments conducted is 85%. The reasons for the wrong identification in this module are similar to those discussed in Section 3.2. As for the inspection of multiple IC, the accuracy obtained is 85%, which is slightly poorer than that in the IC inspection module. This is because some of the features in the IC are lost when the IC image has to be reduced so that more ICs can be inspected at one time. Also, as the camera has to cover a broader area, the lighting has to be even throughout the area under inspection. This is to ensure that the two corners, dynamic windowing and auto-alignment, can work properly for all the ICs to be inspected. Further evaluation is performed to compare between sequential and parallel processing for missing component inspection. It is observed that, for parallel processing, the time taken for inspecting a few components is more than sequential; however, as the number of components to be inspected increases, parallel processing has the advantage. It is also observed that parallel processing shows a small linear increase in time as compared to sequential, which exhibits an exponential increase in timing as the number of components increases. The reason for the above observations is because when fewer components are inspected, the communication overheads in parallel processing are more than in sequential processing. However, when the number of tasks increases, the time taken by the sequential processing increases rapidly. This is because with more tasks, there will be more data and pointers swapping, more tasks to keep track of, etc. Hence,
Transputer-based
automated visual inspection system
Number Fig. 9.
A comparison
of
177
Components
of sequential
versus parallel processing.
the bottleneck in the sequential processing is the speed of the processor. The bottleneck of sequential processing does not exist in the processor farming parallelism. Each worker itself is a processor and the number of workers that can exist in the transputer is limited only by the number of transputers and their memory size. Hence, it can be concluded that the parallel approach is more beneficial as the number of components to be inspected increases. Figure 9 shows the comparison between sequential and parallel processing. In the multiple IC inspection, since the number of pixels occupied by each IC is less, the processing time needed is also less. The accuracy demonstrated is slightly poorer than that of the IC inspection module. This is because some of the features in the IC are lost when the IC image has to be reduced so that more ICs can be inspected at one time.
5 IMPLEMENTATION ASPECTS OF AN INDUSTRIAL TRANSPUTER-BASED VISION SYSTEM The transputer offers a powerful, flexible and cost-effective computer vision hardware for an industrial automated vision system. It is of relatively low cost as compared with other vision system hardware. Its flexibility in its internal architecture enables the transputer to choose the best configuration for any application. Overcoming the bottlenecks in terms of processing capacity and speed can be achieved by simply increasing the number of transputers.
E. K. Teoh, D. P. Mital
178
1
.‘.
-.
MASTER COMPUTER
.VICFoUR
\ .\\
~--cmPlmeus
‘\
Fig. 10.
A transducer-based
automated
visual inspection
I
system.
Figure 10 shows a transputer-based automated visual inspection system set up on an assembly line. Inspection stations are located at two points in the assembly process. The first point is after the automatic component insertion and/or the SMT mounting. Two inspection stations are needed at this point. The first station is to inspect for missing components while the second station inspects for the component type and orientation as well as for improper insertion. The components involved include the IC, diode, resistor and capacitor. The third station for the inspection system is placed after the soldering stage to inspect the solder joint quality. The transputers used in the inspection task use the processor farming method. This is because the image analysis involves tasks that vary in complexity for different components. Hence, the processor farming method is suitable. A host computer is needed to interface with the transputers used for the inspection system, together with the frame grabbers; hence, a computer is needed at each inspection station. These computers are then linked to a master computer to oversee and control the performance of the whole system. To facilitate the inspection of a large PCB, an X-Y table is
Transputer-based
visual inspection
179
recommended for all inspection stations. A two camera set-up is needed for inspection of components as well as improper insertion. The two cameras can be connected to two fransputer-based frame grabbers and connected to the transputers for image capture and analysis. In the solder inspection station, the camera and lighting are placed below the conveyor belt. The inspection area will allow the camera to have an unrestricted view of the solder area; hence, the X-Y table at this stage will have its bottom hollowed out for the camera to view the solder area. Front diffused lighting is used for all the inspection stations-since the lighting over the entire PCB should be as even as possible, this type of lighting is best suited. To overcome bottlenecks in the inspection process, it is proposed that more than one system be set up for the second and third inspection stations as the work involved at these two stations is more intensive. Two stations are shown in Fig. 10. The assembled PCB can be inspected by either station, which are laid out parallel to one another. There is a transition table that will handle and distribute the flow of the assembled PCB for inspection. To improve further on the processing speed of this stage, the integration of a digital signal processor (DSP) into the system can be considered. Preprocessing of the images involved at this stage will improve the reliability of the system. Since most preprocessing (noise removal, etc.) involves a large volume of data to be processed, such a task is best suited for a DSP. Also, some of the algorithms used for inspection will inevitably involve number crunching as well. The DSP can also be used to improve on the processing speed of these algorithms. Hence, low level processing (for example, edge detection) can be handled by the DSP while the transputer can be used to handle medium (for example, edge linking, segmentation, description and recognition) and high level processing (for example, applying artificia1 intelligence techniques), tasks to which a transputer is better suited.
6 CONCLUSION With the increasing popularity of using computer vision to replace manual inspection in industry, the transputer offers a promising option for a cost-effective vision system. In this paper, parallel processing using transputers to achieve real-time image processing for component inspection was explored. Inspection of components at component and board levels was developed using various algorithms. Each algorithm was tailored for each component to achieve accuracy and reliability.
180
E. K. Teoh, D. P. M&al
From the experimental results, inspection of components at a level above 2ooO units h-l has been achieved with high accuracy. Parallel processing in the transputer has also demonstrated its advantage over sequential processing. Hence, with the flexibility in transputer architecture, the transputer does possess the potential for further improvement in real-time applications.
ACKNOWLEDGEMENTS The authors wish to extend their gratitude and appreciation to Professor Brian Lee, Dean of Electrical and Electronic Engineering, Nanyang Technological University, for all his initiatives and support for this research project.
REFERENCES 1. Hata, S., Hagimae, K., Hibi, S. & Gunji, T., Assembled PCB visual inspection machine using image processor with DSP. In Proc. ZECON, pp. 572-7. 2. LeBeau, C. J., Machine vision in semiconductor manufacturing. Advance Imaging Magazine (May) (1991) 54-8. 3. Harp, G., Transputer Applications, 1st edn, Pitman, London, 1989. 4. Schalkoff, R. J., Digital Image Processing and Computer Vision, 1st edn, John Wiley & Sons, New York, 1989. 5. Gonzalez, R. C. & Win&, P., Digital Image Processing, 2nd edn, AddisonWesley, Canada, 1987. 6. Berry, D. T., Colour Recognition using Spectral Signatures, Heriot-Watt University, Edinburgh, UK, 1987, pp. 31-5. 7. Yong, I. N., A robotic vision system with incorporation of artificial intelligence for industrial application, Master Thesis, Nanyang Technological Institute, Singapore, 1988. 8. Kurz, B. J., Optimal quantization for colour displays. IEEE Trans. Pattern Anal. and Machine Intell. (1983) 217-24.