- Email: [email protected]
Flexible Learning Strategies for Visual Control Systems
6a-026
Copyright ~ 1996 IFAC 13th Triennial World Congress, San Francisco, USA
FLEXIBLE LEARNING STRATEGIES FOR VISUAL CONTROL SYSTEMS M. TEUNIS and B. NICKOLA Y
FRAUNHOfER 'I\STITUTI! FOR PRODUCTION SYSTI!MS AND DESIG N TECHNOLOG Y (IPK) .
Pascalstr. 8-9, 10587 Berlin. Germany, e-mail [email protected]
Abstract. Learning image processing systems are a flexible. intelligent inspection tool. Those systems arc based on the experience and know how of its users, and can be adjusted comparatively easy to different product variations forms of error appearance by user oriented information technique. The design and the implementation of a tlexible system learning module is explicated. Through its operation, inspection results will be objective, easier to document and reproducible. Workers who are now doing the inspections could be funher qualified in order to become trainers or instructors of an inspection system. The paper describes the systematic behind learning strategies for human being as well as for machine, and describes the implementation of that knowledge into a visual control system.
aro.
Key Words. Control system design; Automation; image processing; Human factors; computer application
1 INTRODUCTION One problem in all efforts to support human being doing the visual inspection by automatic control systems is the rigidity of these systems. Standard image processing systems for visual inspection, even those of the latest design, generally focus on one special task. Therefore, the system user has been deprived of the tlexibility needed to adjust a visual control system in an uncomplicated manner to new product variations or new designs. Beyond this, the system operator is forced to control the technical system as it is designed. That means, that generally the outline and the operation of conlrol systems are more or less focused on narrow algori thms. and hard-
ware rather than of the way human being is used to carry out a cenain job. Many control systems lack of a user oriented interface, but not only the user interface, also the way a technical system performs (not to say behaves) . That is something that must not necessarily he a problem. The problem is Ihe system design itself. At the Fraunhofer Institute for Production Systems and Design Technology (IPK) in Berlin, Germany a supervised learning visual control system was cb signed. By this approach a system operator is able to teach a visual control system 10 enable the automated recognition of product defects resting upon his/her experiences_ Pattern recognitio n procedures on the
817
basis of supervised learning techniques are the approach. By using the term supervised learning it is understood that the system user is able to train the inspection system easily by presenting each defect that has to be detected automatically -- the system is able to learn by tTaining by shuwing. One main requirement for such a system is the formal description of defoct appearances and theiT correlation to pre.
systems was a first approach to sew a technical inspection system to human beings. The actual image processing -- the object inspection -- is done by the evaluation of so called inspection-windows. A typical size of such windows ranges from 20 to 100 square pixels (by an original image size of 512 square pixels). The inspection parameters like window size and kind offeature extraction & classification methods are fixed. This rigid procedure is linked 10 fixed environmental conditions. This prevents the flexible use of image proct:ssing systems. To get today's learning inspection system ready for the automatic operation. a new approach of three phases is necessary: training. evaluation, and the finai system setup. The actual learning is done during the tnlining & the evaluation.
2 SYSTEM LEARNING To build up a learning system successfully a suitable representation of information, and a suitable choice of knowledge representation formal isms for all kind of information relevant to achieve the system's goal is needed. On the other hand. it is not always easy, or even possible, to select trom the beginning an efioctive representation for a given type of knowledge or inspection behavior. For machine learning, a powerful guideline tor a more effective design strategy of learning systems is starting from the very basic issue of what knowledge should be bandied and how to represent it. It is widely recognized that the effective development of koowledge-based systems cannot avoid to make slTong use of automated means for at least partially acquiring the needed knowledge. Furthermore, to
provide syslems with the ability of self-adapting to environment changes and automatically evolving toward higber level ofperforrnances would make these systems much more attractive to the potential users. A knowledge-intensive process like the visual inspection, a "human-like" approach to machine learning calls for an interdisciplinary cooperation. This js especially because learning does nut only mean discovering regularities, but also to provide explanations and to find out possible causes underlying those regularities. The issue of sequencing effects in knowledge acquisition is fundamenta1 10 design suitable learning strategies. The knowledge of the research in machine learning. which specifies order sensitive algorithms. and the knowledge of psychologists who study the sequence of knowledge acquisition in humans have to come together. Outcomes will be a categorizing of machine learning algorithms. There is a strong need to make data and models accessible in a fonn, knowledge communication, multi-agent learning and problem solving, new approaches to computer based instruction, and training join each other. The design of learning strategies is complementary tu approaches used in the field of neuro-scientific. The training processes, which have to be analyzed, are al a different level than the ones usually xx grabed by neural networks: they are a symbolic abstraction of these. A learning strategy can be viewed as an activity of model building. The different levels in which the symbolic and the neuroscience approaches to learning work, could profilably, interact to provide a more comprehensi ve account of both the learning processes and the knowledge representation mechanisms. What is needed in order to make software more useful and more being used is the capability to adapt it as much as possible to (he system operator.
One way to make the software adaptive is to enable it to learn about the operators inspection behavior. that is. to generate automatically an operators' model. So, what kind of objects and phenomena aTe to be considered as relevant? To answer this question a group of inspection personnel was observed by means of - what mental models of the objects and events during the visual inspection do they use, - what representational formaJisms do they use , and
818
- what is their choice of the specific content of the inspection knowledge.
3 LEARNING STRATEGIES To adapt learning strategies the aim is to emulate decision processes made by the inspection personnel I operator. The operator leads the system to take and record pictures of the object. Then the operator marks areas of interests and is allocating classes to Ihem. All actions taken by the operator arc recorded and analyzed. The stored data describe the image contents as well as decisions recorded from the operator. The aim of the neuronal training is the recreating of the coherency between image contents and the operators representations and classifications. By using the benefits of Fuzzy Logic and Neurofuzzy resources I the approach can be optimi7.ed. The reason for this is obvious: user friendliness will improve and the onler of today is to optimize the man-machine interface where at all possible. Fuzzy logic and Neurofuzzy address such applications as these perfectly as it resembles human decision making with an ability to generate precise solutions from uncertain or approximate information. Fuzzy technology, in the form of approximate reasoning. is also resurfacing in information technology , where it provides decision-support and expert systems with powerful reasoning capabilities, bound by a minimum of rules. Of course, fuzzy logic is not the best approach for every control problem. As designers look at ils power and expressiveness. they must decide where to apply it, and also how best to manage software projects based on this technology.
one formalism to another one. To avoid such backwards. the following scenario has been chosen: Given a target behavior in a chosen inspection job. all the knowledge required to perform the job has to be selected and repre.ented (in the context of this paper, the inspection job is also called task). Selection and representation are not one-step activities: on the contrary, the current choices can be examined. modified, and extended any time (Fig I). In order to make this dynamic adaptation possible, two main challenges have to be faced, inspection window
,
,
:~ .,. ,.,
..-
'. " .
a b a b Fig. I. Task represenlalion for creating learning sets. By moving inspection windows from preset posilions into areas of interests, the change of features (a Histogram, b Co.Occurrence-Matrix) will be recorded. 3. J Examination and represefl1lltions
choosing
of
knowledge
In order to encourage possible changes in a representaOne problem by designing learning strategies is still that there are almosl always not enough training daJ:a to cover the real world case. The so called learning behavior relies heavily on representative leaning data sets. It is important to understand why, when and how changes in representations, at every level, occur. Most machine learning systems currently use a single representation formalism , such as, logics or semantic nets or conceptual dependency graphs. The problem of these systems is, that they seem unable to move from
I Fuzzy Logic MU Neurofuzzy re~ourCes became very popular. nO( to say rashioned. lnere are many apptical.toos that lake bene· fits from these approaches. but there are still a lot of applications. where the "old· fashioned way" based on statistical algorithms is
tion, the adequacies of it have to be establiSh, followed by an updating strategy to be selected. The first issue concerns the evaluation, like the definition of criteria [Q a~sess similarities or differences between the defect free areas of inspection objects and the behavior obtained by the inspection personnel using the current representation schemes. The second issue involves a decision: what learning strategy does the comparison results between the target (final system setup) arxi current behaviors of the inspection personnel suggest? Several possibilities are possibly: one, can be not to learn , fore instance to leave things a~ they are. Another one, to try to make narrow changes sufficient is to account for the new fragment of information.
~UperiOl .
819
3.2 Specific relations between Ihe ill.fpeclion job alli representation of objects It is obvious to mention, that humans are competent at selecting the correct way of representing knowledge for a given task. Trying to make specific relationships between the inspection job and the object representations could lead la substantial improvements in the design of the learning inspection system. To approach this problem, two aspects are to be considered: how can a ,ask be represented and, what are tbe characteristics of a representation scheme that make it suitable for achieving a target behavior for the automated inspection'!
must decompose the condition-effect causal link into two segments: different conditions of collaboration create different interaction process.e.~ and different interaction processes lead to different learning effects, 1be designed model supports the implementation in multiagent systems_ The goal of distributed artificial intelligence software is to implement networks of agents able to solve together problems that are beyond their individual abllities.. In this context agents can be described as computer programs which operate autonomous.
4 SYSTEM TRAINING AND APPLICA TION
A learning process usually involves different types of knowledge. If onc worker has different representation forms for his knowledge, then problem solving during the visual inspection consists of two types of processes: transfonning the information contained in one repr~entatioll into the other and backwards, and applying knowledge that is available in the used representation. Under such circumstances it is probable that over ti me a worker has developed different more or less related representations for the same inspection job. One representation that enables to do expert problem solving diagnosis and one that covers the knowledge about (he relations and strategic knowledge about how the inspection is done correctly.
The object to be inspected must be illuminated properly and then rerorded in the appropriated piKel resolution . Then the inspection windows must be set within the image. The areas covered by inspection windows must be representative for the defined inspection classes. The selected image areas will be processed with chosen image processing algorithms, and the calculated features are stored in a data base. The setup parameters are generally leaned on the algorithms of the image processing software. This procedure must be carried out for each inspection class.
Realizing that solving a task. with a single representation does not work effectively. the learning goal would be to find an alternat.i ve representation and to translate the problem to that one - and back. Realizing that while using different. representations for one task the coordination becomes difficult or impossible. The goal of the learning strategy mus( be to tind ways of improving the coordination. The conception of goa1oriented learning i.'i chosen as an approach to ana1yze multiobjecti ve learning with multiple representations. The purpose is to achieve precise symbolic representations of human learning strategies. The chosen machine learning strategy is able to learn Incrementally si nce it is never possible for a computer system to hold arbitrary amounts of information in the memory for postponed analysis.
An ideal human oriented system set up would be an entirely simulation of the (mentally) way a worker is dOing the inspection. The described procedure has some problems in case - the inspection object can not be presented to the camera system within the same location, - the inspection areas can not be covered by squared windows, - the defect inspection requires different cla~sification methods, and - the operator is not capable to train the system appropriate. The new approach is to translate the input given by the operator into paramecers the image processing software is able to operate. The input is not adapted to the systems requirements. but 10 the way , the operator is used to carry out her/his job. Therefore the new flexible learning interface transfonns the fuzzy input into precise parameters -- the inspection behavior of human being will be transfonned into system func-
To understand how the described conditions interact with each other and how they affect learning. one
820
tions and pammeters. The methods, roquired for the system training, are rooted on different approaches (Lea rning Classifiers. Genetic Algorithms, NeuroFUI.ly-Approches, Evolutionary Strategies, etc. ), For themselves, they are already available as engineering tools or they are pan of the ongoing research. Altogether, they do represent more or less the state of the art in R&D. To get flexible learning strategies to work, these methods have to be linked. Those links are based on: - infonnation who are recei ved by fuzzy inputs about the inspection job to be carry out automatically by the control system, - knowledge about subjective experiences of the workers doing the inspection job manually , - o bjective knowledge, - an existing data base of image processing algorithms as well as rules about links between those algorithms. As already mentioned, for a final set up of a learning vi sual control system, a training stage is needed. Such a Iraining has to be carry out under two point~ of view: • training of object models and tolerances (kind. shape, and appearance of product features to be re. tected), • learning of system procedures (panuneters for image processing algorithms. feature extraction, defect recognition). The learning of the system itself occurs with the support of supervised as we ll as non supervised training stages. It is possible to have a couple of cycles of supervised and non supervised training stages, if the job requires so. Therefore these stages are structured in a way, that interactions within all stages with all stages are possible. That enables dependent of intermediate results - the transfer and modification of system parameters to be modified during the entire learning phase. The supervised training is done by relatively si mple statements and ratings by a skilled operator. The operator must have no detailed knowledge about the image processing procedures themselves. 1be manual inputs consist for instance of: defect visibility (sufficient contrast defect areas/defect free areas within the input image),
a general mark of the inspection area~ (within the input image). a mark of faulty areas settings, statements, what kind of defects will be located in what area, statements about the automaticall y generated inspection area segmentation ("inspection area. correspond to the requirement", "location of area borders have to be improved"), statements about the re s ult~ of the automatic defect recognilion ("defect recognition is sufficient", "defect recognhion has to be improved". "the , improvements have to occur into a certain direction: area segmentation, defect location, etc.").
According to the basic adjustments of the supervised training, the result of the non-supervised training is an optimization of problem specific inspection procedures. This is done mainly by an automatic run, driven by rules of image interpretation methods. Finally it is wise to a:id a supervised training slage again to confirm the automatic run. For both training procedures, the flexible learning strategy is based on an entirely evaluation of a-prioriknowledge. This includes components of objective as well as subjective knowledge components. 1be knowledge is first of all a collection/summary of the experiences made by a wide range of workers, who were in charge for the visual inspection. The experiences are represented in different ways sllch as: models for different layers within the recognition process, rules about the selection of the order of the classification steps, knowledge basis about defec t shapes and appearance and about recognition procedures, knowledge basis about the segmentation of the defect areas, CAD data. The creatio n of a problem specific inspection run - the final setup - is based on fuzzy uperator inputs, is typically build up as follows: 1. Detection of the geometric shape and location of the inspection areas (areas of interests - AD]).
821
2. Detection of suitable inspection windows. which support the recognition of the respective product deti:clS.
3. Selection/configuration of suitable recognition procedures within the respective inspection area.
Kubat, M. (1993). Machine Learning. In: Advanced Topics in artificial Intelligence, International Summer School. Prague, Springer-Verlag, Berlin. Michalski, R.S. (1991). Learning from Observation: Conceptual Clustering. In: Machine Learning, Morgan Kaufmann.
4. Parametration of the recognition procedure on
basis of simple quality criteria (robust recognition rate, high recognition speed).
5 CONCLUSION The challenge to design a visual inspection system based on flexible learning strategies is to bundle up with wide procedural and declarative knowledge bases. These methods are based on techniques like supervised learning, Neuronal Nets, Evolutionary Programming. Texture Analysis. Morphological Image Processing. Rule Based Segmentation . Most important for the
parametration and optimization is the design of the user interface. The purpose by designing the user interface is to decrease (he work load of the inspection personnel and to ex.pand the communication skills and the understanding of production processes ao;; well. 1he development includes tasks in the field of technique. organization and personn.el as well. To be able to design flexible learning strategies for machine learning. there is a strong need to understand how human beings are used to learn and how to they are used to present their knowledge, they already have adopted. A successful introduction of a visual control system based on flexible Iraining modules relies heavily on the experience and knowledge of human being.
6 REFERENCES Khan, E. Neufuz (1993). An Intelligent Combination of Fuzzy Logi c with Neural Nets, Proceedings of International Joint Conference on Neural Networks. Nagoya, JP, 1993. S. 2945-2950 Riecken , D. (1994). Intelligent agents. CA CM, 37(7), 18-21). Haralik, R. M. (1987). Statistical Image and Texture Analysis. Handbook of Pattern Recognition arl Images Processing: Academic Press San Diego.
822
Copyright ~ 1996 IFAC 13th Triennial World Congress, San Francisco, USA
FLEXIBLE LEARNING STRATEGIES FOR VISUAL CONTROL SYSTEMS M. TEUNIS and B. NICKOLA Y
FRAUNHOfER 'I\STITUTI! FOR PRODUCTION SYSTI!MS AND DESIG N TECHNOLOG Y (IPK) .
Pascalstr. 8-9, 10587 Berlin. Germany, e-mail [email protected]
Abstract. Learning image processing systems are a flexible. intelligent inspection tool. Those systems arc based on the experience and know how of its users, and can be adjusted comparatively easy to different product variations forms of error appearance by user oriented information technique. The design and the implementation of a tlexible system learning module is explicated. Through its operation, inspection results will be objective, easier to document and reproducible. Workers who are now doing the inspections could be funher qualified in order to become trainers or instructors of an inspection system. The paper describes the systematic behind learning strategies for human being as well as for machine, and describes the implementation of that knowledge into a visual control system.
aro.
Key Words. Control system design; Automation; image processing; Human factors; computer application
1 INTRODUCTION One problem in all efforts to support human being doing the visual inspection by automatic control systems is the rigidity of these systems. Standard image processing systems for visual inspection, even those of the latest design, generally focus on one special task. Therefore, the system user has been deprived of the tlexibility needed to adjust a visual control system in an uncomplicated manner to new product variations or new designs. Beyond this, the system operator is forced to control the technical system as it is designed. That means, that generally the outline and the operation of conlrol systems are more or less focused on narrow algori thms. and hard-
ware rather than of the way human being is used to carry out a cenain job. Many control systems lack of a user oriented interface, but not only the user interface, also the way a technical system performs (not to say behaves) . That is something that must not necessarily he a problem. The problem is Ihe system design itself. At the Fraunhofer Institute for Production Systems and Design Technology (IPK) in Berlin, Germany a supervised learning visual control system was cb signed. By this approach a system operator is able to teach a visual control system 10 enable the automated recognition of product defects resting upon his/her experiences_ Pattern recognitio n procedures on the
817
basis of supervised learning techniques are the approach. By using the term supervised learning it is understood that the system user is able to train the inspection system easily by presenting each defect that has to be detected automatically -- the system is able to learn by tTaining by shuwing. One main requirement for such a system is the formal description of defoct appearances and theiT correlation to pre.
systems was a first approach to sew a technical inspection system to human beings. The actual image processing -- the object inspection -- is done by the evaluation of so called inspection-windows. A typical size of such windows ranges from 20 to 100 square pixels (by an original image size of 512 square pixels). The inspection parameters like window size and kind offeature extraction & classification methods are fixed. This rigid procedure is linked 10 fixed environmental conditions. This prevents the flexible use of image proct:ssing systems. To get today's learning inspection system ready for the automatic operation. a new approach of three phases is necessary: training. evaluation, and the finai system setup. The actual learning is done during the tnlining & the evaluation.
2 SYSTEM LEARNING To build up a learning system successfully a suitable representation of information, and a suitable choice of knowledge representation formal isms for all kind of information relevant to achieve the system's goal is needed. On the other hand. it is not always easy, or even possible, to select trom the beginning an efioctive representation for a given type of knowledge or inspection behavior. For machine learning, a powerful guideline tor a more effective design strategy of learning systems is starting from the very basic issue of what knowledge should be bandied and how to represent it. It is widely recognized that the effective development of koowledge-based systems cannot avoid to make slTong use of automated means for at least partially acquiring the needed knowledge. Furthermore, to
provide syslems with the ability of self-adapting to environment changes and automatically evolving toward higber level ofperforrnances would make these systems much more attractive to the potential users. A knowledge-intensive process like the visual inspection, a "human-like" approach to machine learning calls for an interdisciplinary cooperation. This js especially because learning does nut only mean discovering regularities, but also to provide explanations and to find out possible causes underlying those regularities. The issue of sequencing effects in knowledge acquisition is fundamenta1 10 design suitable learning strategies. The knowledge of the research in machine learning. which specifies order sensitive algorithms. and the knowledge of psychologists who study the sequence of knowledge acquisition in humans have to come together. Outcomes will be a categorizing of machine learning algorithms. There is a strong need to make data and models accessible in a fonn, knowledge communication, multi-agent learning and problem solving, new approaches to computer based instruction, and training join each other. The design of learning strategies is complementary tu approaches used in the field of neuro-scientific. The training processes, which have to be analyzed, are al a different level than the ones usually xx grabed by neural networks: they are a symbolic abstraction of these. A learning strategy can be viewed as an activity of model building. The different levels in which the symbolic and the neuroscience approaches to learning work, could profilably, interact to provide a more comprehensi ve account of both the learning processes and the knowledge representation mechanisms. What is needed in order to make software more useful and more being used is the capability to adapt it as much as possible to (he system operator.
One way to make the software adaptive is to enable it to learn about the operators inspection behavior. that is. to generate automatically an operators' model. So, what kind of objects and phenomena aTe to be considered as relevant? To answer this question a group of inspection personnel was observed by means of - what mental models of the objects and events during the visual inspection do they use, - what representational formaJisms do they use , and
818
- what is their choice of the specific content of the inspection knowledge.
3 LEARNING STRATEGIES To adapt learning strategies the aim is to emulate decision processes made by the inspection personnel I operator. The operator leads the system to take and record pictures of the object. Then the operator marks areas of interests and is allocating classes to Ihem. All actions taken by the operator arc recorded and analyzed. The stored data describe the image contents as well as decisions recorded from the operator. The aim of the neuronal training is the recreating of the coherency between image contents and the operators representations and classifications. By using the benefits of Fuzzy Logic and Neurofuzzy resources I the approach can be optimi7.ed. The reason for this is obvious: user friendliness will improve and the onler of today is to optimize the man-machine interface where at all possible. Fuzzy logic and Neurofuzzy address such applications as these perfectly as it resembles human decision making with an ability to generate precise solutions from uncertain or approximate information. Fuzzy technology, in the form of approximate reasoning. is also resurfacing in information technology , where it provides decision-support and expert systems with powerful reasoning capabilities, bound by a minimum of rules. Of course, fuzzy logic is not the best approach for every control problem. As designers look at ils power and expressiveness. they must decide where to apply it, and also how best to manage software projects based on this technology.
one formalism to another one. To avoid such backwards. the following scenario has been chosen: Given a target behavior in a chosen inspection job. all the knowledge required to perform the job has to be selected and repre.ented (in the context of this paper, the inspection job is also called task). Selection and representation are not one-step activities: on the contrary, the current choices can be examined. modified, and extended any time (Fig I). In order to make this dynamic adaptation possible, two main challenges have to be faced, inspection window
,
,
:~ .,. ,.,
..-
'. " .
a b a b Fig. I. Task represenlalion for creating learning sets. By moving inspection windows from preset posilions into areas of interests, the change of features (a Histogram, b Co.Occurrence-Matrix) will be recorded. 3. J Examination and represefl1lltions
choosing
of
knowledge
In order to encourage possible changes in a representaOne problem by designing learning strategies is still that there are almosl always not enough training daJ:a to cover the real world case. The so called learning behavior relies heavily on representative leaning data sets. It is important to understand why, when and how changes in representations, at every level, occur. Most machine learning systems currently use a single representation formalism , such as, logics or semantic nets or conceptual dependency graphs. The problem of these systems is, that they seem unable to move from
I Fuzzy Logic MU Neurofuzzy re~ourCes became very popular. nO( to say rashioned. lnere are many apptical.toos that lake bene· fits from these approaches. but there are still a lot of applications. where the "old· fashioned way" based on statistical algorithms is
tion, the adequacies of it have to be establiSh, followed by an updating strategy to be selected. The first issue concerns the evaluation, like the definition of criteria [Q a~sess similarities or differences between the defect free areas of inspection objects and the behavior obtained by the inspection personnel using the current representation schemes. The second issue involves a decision: what learning strategy does the comparison results between the target (final system setup) arxi current behaviors of the inspection personnel suggest? Several possibilities are possibly: one, can be not to learn , fore instance to leave things a~ they are. Another one, to try to make narrow changes sufficient is to account for the new fragment of information.
~UperiOl .
819
3.2 Specific relations between Ihe ill.fpeclion job alli representation of objects It is obvious to mention, that humans are competent at selecting the correct way of representing knowledge for a given task. Trying to make specific relationships between the inspection job and the object representations could lead la substantial improvements in the design of the learning inspection system. To approach this problem, two aspects are to be considered: how can a ,ask be represented and, what are tbe characteristics of a representation scheme that make it suitable for achieving a target behavior for the automated inspection'!
must decompose the condition-effect causal link into two segments: different conditions of collaboration create different interaction process.e.~ and different interaction processes lead to different learning effects, 1be designed model supports the implementation in multiagent systems_ The goal of distributed artificial intelligence software is to implement networks of agents able to solve together problems that are beyond their individual abllities.. In this context agents can be described as computer programs which operate autonomous.
4 SYSTEM TRAINING AND APPLICA TION
A learning process usually involves different types of knowledge. If onc worker has different representation forms for his knowledge, then problem solving during the visual inspection consists of two types of processes: transfonning the information contained in one repr~entatioll into the other and backwards, and applying knowledge that is available in the used representation. Under such circumstances it is probable that over ti me a worker has developed different more or less related representations for the same inspection job. One representation that enables to do expert problem solving diagnosis and one that covers the knowledge about (he relations and strategic knowledge about how the inspection is done correctly.
The object to be inspected must be illuminated properly and then rerorded in the appropriated piKel resolution . Then the inspection windows must be set within the image. The areas covered by inspection windows must be representative for the defined inspection classes. The selected image areas will be processed with chosen image processing algorithms, and the calculated features are stored in a data base. The setup parameters are generally leaned on the algorithms of the image processing software. This procedure must be carried out for each inspection class.
Realizing that solving a task. with a single representation does not work effectively. the learning goal would be to find an alternat.i ve representation and to translate the problem to that one - and back. Realizing that while using different. representations for one task the coordination becomes difficult or impossible. The goal of the learning strategy mus( be to tind ways of improving the coordination. The conception of goa1oriented learning i.'i chosen as an approach to ana1yze multiobjecti ve learning with multiple representations. The purpose is to achieve precise symbolic representations of human learning strategies. The chosen machine learning strategy is able to learn Incrementally si nce it is never possible for a computer system to hold arbitrary amounts of information in the memory for postponed analysis.
An ideal human oriented system set up would be an entirely simulation of the (mentally) way a worker is dOing the inspection. The described procedure has some problems in case - the inspection object can not be presented to the camera system within the same location, - the inspection areas can not be covered by squared windows, - the defect inspection requires different cla~sification methods, and - the operator is not capable to train the system appropriate. The new approach is to translate the input given by the operator into paramecers the image processing software is able to operate. The input is not adapted to the systems requirements. but 10 the way , the operator is used to carry out her/his job. Therefore the new flexible learning interface transfonns the fuzzy input into precise parameters -- the inspection behavior of human being will be transfonned into system func-
To understand how the described conditions interact with each other and how they affect learning. one
820
tions and pammeters. The methods, roquired for the system training, are rooted on different approaches (Lea rning Classifiers. Genetic Algorithms, NeuroFUI.ly-Approches, Evolutionary Strategies, etc. ), For themselves, they are already available as engineering tools or they are pan of the ongoing research. Altogether, they do represent more or less the state of the art in R&D. To get flexible learning strategies to work, these methods have to be linked. Those links are based on: - infonnation who are recei ved by fuzzy inputs about the inspection job to be carry out automatically by the control system, - knowledge about subjective experiences of the workers doing the inspection job manually , - o bjective knowledge, - an existing data base of image processing algorithms as well as rules about links between those algorithms. As already mentioned, for a final set up of a learning vi sual control system, a training stage is needed. Such a Iraining has to be carry out under two point~ of view: • training of object models and tolerances (kind. shape, and appearance of product features to be re. tected), • learning of system procedures (panuneters for image processing algorithms. feature extraction, defect recognition). The learning of the system itself occurs with the support of supervised as we ll as non supervised training stages. It is possible to have a couple of cycles of supervised and non supervised training stages, if the job requires so. Therefore these stages are structured in a way, that interactions within all stages with all stages are possible. That enables dependent of intermediate results - the transfer and modification of system parameters to be modified during the entire learning phase. The supervised training is done by relatively si mple statements and ratings by a skilled operator. The operator must have no detailed knowledge about the image processing procedures themselves. 1be manual inputs consist for instance of: defect visibility (sufficient contrast defect areas/defect free areas within the input image),
a general mark of the inspection area~ (within the input image). a mark of faulty areas settings, statements, what kind of defects will be located in what area, statements about the automaticall y generated inspection area segmentation ("inspection area. correspond to the requirement", "location of area borders have to be improved"), statements about the re s ult~ of the automatic defect recognilion ("defect recognition is sufficient", "defect recognhion has to be improved". "the , improvements have to occur into a certain direction: area segmentation, defect location, etc.").
According to the basic adjustments of the supervised training, the result of the non-supervised training is an optimization of problem specific inspection procedures. This is done mainly by an automatic run, driven by rules of image interpretation methods. Finally it is wise to a:id a supervised training slage again to confirm the automatic run. For both training procedures, the flexible learning strategy is based on an entirely evaluation of a-prioriknowledge. This includes components of objective as well as subjective knowledge components. 1be knowledge is first of all a collection/summary of the experiences made by a wide range of workers, who were in charge for the visual inspection. The experiences are represented in different ways sllch as: models for different layers within the recognition process, rules about the selection of the order of the classification steps, knowledge basis about defec t shapes and appearance and about recognition procedures, knowledge basis about the segmentation of the defect areas, CAD data. The creatio n of a problem specific inspection run - the final setup - is based on fuzzy uperator inputs, is typically build up as follows: 1. Detection of the geometric shape and location of the inspection areas (areas of interests - AD]).
821
2. Detection of suitable inspection windows. which support the recognition of the respective product deti:clS.
3. Selection/configuration of suitable recognition procedures within the respective inspection area.
Kubat, M. (1993). Machine Learning. In: Advanced Topics in artificial Intelligence, International Summer School. Prague, Springer-Verlag, Berlin. Michalski, R.S. (1991). Learning from Observation: Conceptual Clustering. In: Machine Learning, Morgan Kaufmann.
4. Parametration of the recognition procedure on
basis of simple quality criteria (robust recognition rate, high recognition speed).
5 CONCLUSION The challenge to design a visual inspection system based on flexible learning strategies is to bundle up with wide procedural and declarative knowledge bases. These methods are based on techniques like supervised learning, Neuronal Nets, Evolutionary Programming. Texture Analysis. Morphological Image Processing. Rule Based Segmentation . Most important for the
parametration and optimization is the design of the user interface. The purpose by designing the user interface is to decrease (he work load of the inspection personnel and to ex.pand the communication skills and the understanding of production processes ao;; well. 1he development includes tasks in the field of technique. organization and personn.el as well. To be able to design flexible learning strategies for machine learning. there is a strong need to understand how human beings are used to learn and how to they are used to present their knowledge, they already have adopted. A successful introduction of a visual control system based on flexible Iraining modules relies heavily on the experience and knowledge of human being.
6 REFERENCES Khan, E. Neufuz (1993). An Intelligent Combination of Fuzzy Logi c with Neural Nets, Proceedings of International Joint Conference on Neural Networks. Nagoya, JP, 1993. S. 2945-2950 Riecken , D. (1994). Intelligent agents. CA CM, 37(7), 18-21). Haralik, R. M. (1987). Statistical Image and Texture Analysis. Handbook of Pattern Recognition arl Images Processing: Academic Press San Diego.
822