- Email: [email protected]
Genetic Algorithms for Automatic Design of Fuzzy Decision Systems for Intelligent Manufacturing
Copyright ~ IFAC Intelligent Components and Instruments for Control Applications, Annecy, France, 1997
GENETIC ALGORITHMS FOR AUTOMATIC DESIGN OF FUZZY DECISION SYSTEMS FOR INTELLIGENT MANUFACTURING
EGRESITS, CS.; MONOSTORI, L.
Computer and Automation Research Institute, Hungarian Academy ofSciences Kendeu.13-17. H-1518 Budapest, POB63, Hungary Tel: (+36 1) 1665-644, Fax: (+36 1) 1667-503, E-mail: [email protected]
Abstract: Artificial neural networks (ANNs) have been successfully applied in different fields of manufacturing. Monitoring and modelling of manufacturing processes obviously belong to the most promising areas, where real-time nature, uncertainty handling and learning abilities are essential. However, mainly because of the "black box" nature of ANNs, these solutions have limited industrial acceptance. In the paper, a combined use of the neural and fuzzy techniques in cutting tool monitoring is illustrated. We introduce a genetic algorithm based approach to overcome problems, including rule selection and redundancy checking. Finally the results are compared with ANN and previous neurofuzzy (NF) approaches.
Keywords: genetic algorithm, neuro-juzzy systems, intelligent manufacturing
1. INTRODUCTION
• to introduce a genetic algorithm based approach to overcome these problems, including: => selection of important fuzzy rules in case of large number of linguistic variables and MBFs => checking the redundancies in rule base => using fuzzy rules incomplete in precondition site to simplify and reduce rule set size • to make comparisons between the back propagation ANNs, the Lin-Lee algorithm and the suggested solution, on well known classification problems, • demonstrate the applicability of GA supported neuro-fuzzy approach for monitoring and control of manufacturing processes
Fuzzy systems are considered to be a natural link between symbolic and subsymbolic approaches. On the one hand they can work in real time circumstances and handle uncertainties as artificial neural networks, on the other hand can manage both symbolic and numerical information. However it is very hard to identify the fuzzy rules and tune membership functions (MBF) of the fuzzy reasoning. It seems that the best performance can be prospectively obtained by combining neural and fuzzy approaches, integrating their benefits. The resulting neuro-juzzy system (NF) is hybrid system, where the system architecture remain fuzzy, but using neural learning techniques, it can be trained automatically using neural and genetic learning techniques. The aims of the paper are the following: • to describe and analyse a frequently used NF model introduced by Lin-Lee [5], with special emphasis on difficulties of rule selection
2. THE LIN-LEE NEURO-FUZZY MODEL In this section, the NF model used in the investigations described later in the paper is introduced. The model basically follows the main
551
objectives of the solution described by Lin-Lee. The system consist of five layers (Figure I .). The first layer is the input layer. Each node in this layer represents a linguistic variable (F ,-F n). The second and the fourth layer contain tenn nodes which act as MBF to represent the tenns of the respective linguistic variable. The third layer is the rule node layer, where each node with its connection represents a fuzzy rule. Layerl
Layer2
Layer3
(lnP!Jf
(input Iwnn
n~)
linguistic nOdes)
_)
(
I
Layer4 (output term
nodu)
In this model, the selection of important fuzzy rules proceeds by competitive learning, which requires the generation of the whole structure at the beginning of the learning process, i.e. the net with all possible rules . That means. the third layer has as many elements, as the product of the number of different MBFs assigned to each input and output variables. This approach leads to a combinatorial explosion. This neural learning is suitable for building fuzzy logic systems with relative small number of linguistic variables. For some applications, where the number of input and output variables is large, this will create a large amount of fuzzy rules and regions which will cause learning difficulties, huge memory and hardware implementation problems. An other drawback to be mentioned is that the subset of "best" rules selected by the competitive phase is not necessarily the "best" subset of rules. The main goal of the solution described in this paper is to eliminate these shortcomings by using genetic algorithms (GA) for rule generation.
layerS (_
ilngu)stic nOdes)
~~~~~
3. GE,.'ETIC ALGORITHMS Genetic algorithms are inspired by processes that appear in biological evolution and the working of the immune system. GAs use a direct analogy of natural behaviour and work with a population of "individuals". Each individual is a potential solution of a given problem space and assigned a fitness value which measure its goodness. The fittest individuals get more opportunities to reproduce themselves. A new individual is created either by crossover or mutation operators. The fonner combines genetic infonnation of randomly chosen "parents", the latter randomly modifies parts of the infonnation describing an individual. Mutation provides a small amount of random search, and helps ensure that no point in the search space has a zero probability of getting into the population . After the randomly initialised population, over many generations, the populations evolve and could give better and better individuals [I] . Genetic algorithms are very robust due to the global searching. However, as it has been widely recognised, the GA involves intensive computation and is less efficient compared with other machine learning techniques.
Figure I . 5-layer structure of the implemented neurofuzzy model (The arrows represent signal flow in forward direction) Layer three links define the preconditions of the rules, and layer-four links incorporate the rules' consequences. Bell shaped membership functions were used in the investigations with the mean mi,j , variance fJi,j of the jth tenn of the ith input linguistic variable Fi. Perfonning precondition matching of fuzzy rules, the rule nodes in layer 3 fulfil fuzzy AND (min) operation. The nodes in layer 4 integrate the fired rules having the same consequence by a fuzzy OR (sum) operator. In forward pass layer 5 perfonns center of area defuzzification The system can be constructed from the training examples with hybrid learning schemes. The algorithm suggested by Lin and Lee consist of four consecutive steps:
4. A GA SOLUTION FOR SELECTING FUZZY RULES
• Determination of membership functions by selforganised clustering, • Selection of the most important fuzzy rules by competitive learning, • Elimination and combination of rules, • Adjustment of the parameters of the MBFs by supervised back propagation learning.
4.1 Representation A genetic algorithm can be used to discover a desirable optimal set of rules. To apply GAs, rules should be genetically coded and we need to define
552
appropriate fitness function and genetic operators. In the proposed approach instead of using a binary string, we use a integer number string (a vector) to represent a particular case or object. An individual of population is interpreted by a fuzzy rule set which is coded to a string (Figure 2).
Spreading of empty rules results in decreasing the number of rules in a rule set. Usual NF models operate well when in a rule all precondition exist, but generating rule set \\ith incomplete rules or simplifying and reducing rule set cause difficulties. In this representation introducing incomplete rules is a trivial task. We only allow "0" characters in place of any aJj . Moreover, this solution expands the search-space substantially.
R/=2 ...24 R J= 3 ... 1 1
4 ... 32 '-.t-'
4.2 Operators
~
The calculation of a new generation P(t+ J) from P(t) is done by sequentially applying the operators selection, crossover and mutation (Figure 3). The most fundamental genetic operator is the selection, in our model roulette-wheel selection was applied where the probability of selection of a specific string is proportional to its strength [I] .
R,= 4...32
Population N
Population N + I
F,,,;
Prec:oaditiolls
..
......Crossover
ConsequeDcc
""'"'""R.....,....... . . 24
~~
Rr
Mutation
r
Figure 2. Representation and genetic coding of the neuro-fuzz}' rules
Ll 1
R
Jl.r.=4 . .3 '2
Let's consider the case with n input variables (F 1Fn) and one output variable (01)' A rule has the following form: IF Fl is middle and F2 is high .. . Fn is low THEN 01 is weak, where middle, high, etc. are MBFs of the corresponding variables. In the chosen coding, the MBFs of the kth input or output variable are one by one mapped to natural numbers in interval [l ,2, ... jk.l, where jk denotes the number of all possible MBFs of the given variable. A rule set consists of maximum r pieces of rules and an individual (rule set) is coded to a string:
Figure 3. Calculation of new generation in applied Genetic algorithm Crosso\'er regarded as the most powerful genetic operator and the main engine for exploration in GAs. This is in touch \\ith the building block hypothesis, as it is the operator responsible for shuffling and recombination of building blocks. R,=!..14 R,=2.,.ll R,-2..22
R,- !...l J R,=: 11 R J ::;:; ; : ;
. . . . . . ...
where a/i and bl are the mapped values of the corresponding MBFs of the ith input and the output variable respectively, in Ith rule. For example if the FI input variable has four membership functions very_high, high, low and very_low, and FI is high in the ith rule then ai ]'=2. A population contains strings 'with the same length. A special rule named "empty" rule is introduced in the representation which basn't got any preconditions and consequences. Every rule has the same length, therefore the empty rule is coded with n+ I (number of inputs + number of outputs ) "0" characters.
R. 1=3 J 3
Parent 1
R, -1 .,3 ~
R...,-3 .. .33 R, =1...34
Parent 2
Child 1
Cbild 2
Figure 4. Single point crossover The most simple crossover takes two selected individuals and cuts their chromosome strings at a
553
randomly chosen position only at the bounds of rules producing two "heads" and two "tails". The two offsprings come into existence from parents changing their tails (Figure 3). In the initial stage to get more genetic mix ing we used uniform crossover because the probability of finding building blocks is munch lower and the convergence is quicker (Figure 4). Uniform crossover radically different to one point crossover. Each gene in the offspring is created by copying the corresponding gene from one or the other parent chosen according to a randomly generated crossover mask. Crossover isn't applied to all pairs, usual probability of crossover is 0.6-0.8. After every crossover the redundant rules in a rule set are replaced by an empty rule, so no more than one copy remains from a given type of rule .
We investigated effects of the GAs parameters and comes to the conclusion that as a rule of thumb, the crossover rate or probability should be about 0.6, and mutation rate should be about 0.005 . The system is not sensitive to small changes to crossover rate and using 0.005 gives good learning performance even when crossover rate is zero. To preserve the best individual of the nth generation from being changed by the genetic operators, this individual might be copied to the next generation. This strategy called fittest survive does not always increased the performance of the algorithm. 4.3 Fitness function
In this case the fitness function is a weighted quadratic error of the rule set counted as the sum of quadratic errors of the training patterns multiplied by recognition rate of the system . This weight is related to the number of non-empty rules of rule set and assists spreading of empty rules for reduce of rule set. For fitness calculation the system has to generate the rules for the net and run the classification process.
Crossover mask
R,· 1... 1 •
R, . ] . 14
. . . ..
~L,u,
R:·2: J I R,·2 .22:
..... ~
R _¥;fJ R.· I ,.l4 Parent 2
Parent 1
Child 1
Child 2
5. EXPERIMENTS Figure 5. Multi point uniform crossover
We developed in the Computer and Automation Research Institute, Budapest an ANN and neurofuzzy simulator caIled NEURECA . It was written in C++ using its object oriented nature enabling to dynamicaIly vary the network structure during learning and to implement different ANN models including various neuro-juzzy approaches. NEURECA provides the following main features in an integrated framework: • definition of different statistical and spectral features for various channels • on-line feature computation, • automatic feature selection, • manipulation , visualization of pattern files . • ANN learning with back propagation (BP) algorithm , • different neuro fuzzy learning techniques including whole Lin-Lee model • classification, estimation of unknown patterns, • standardized (DOE) interfaces to other programs, etc. [4] The back propagation (BP) part of the system was successfuIly applied for different problems (e.g. first column of the Table 1). However, ANN models have a set of problems: • lengthy training times. • dependence on the initial parameters, • lack of a problem-independent way to choose appropriate network topology, • incomprehensive (black box) nature of A1'-.'Ns .
Individuals become very similar before a nearly optimal solution is reached, thus preventing any further progress. Mutation acts against this, by constantly generating new rules, thus prevent the population from getting trapped in a local maximum in the search space. However, the action of mutation can sometimes result in the loss of good individuals, so the need to prevent convergence has to be balanced against the inevitable loss of efficiency due to the disruption of good genetic material. Mutation is applied to each child individuaIly after crossover. It randomly modifies each character of the code with a small probability (Figure 6).
R20/d= ..
R I=2 ...2 I Rl=2 ...22
,J
2 ... 22
~. 1 2 3
~
R.= 2 .. 3 I
~
=
Candidates
R 2new= 2 ... 3 2 •. ...
random selection
R I = 2.2 I R1= 2... 3 2
R.= 2 .. 3 I
Figure 6. Mutation
554
XOR with 3 input
Two spiral
Iris
ANN 8ackpropagation 100% 25 sec 350 iteration 100% 4,5 hours 95000 iteration 100% 2,5 hours 68000 iteration
Milling tool classification with 6 inputs
100% 2 min 15 sec
Milling tool classification with 10 inputs
100% 5 min 30 sec
Lin-Lee Neuro-Fuzzy Model
GA supported Neuro-Fuzzy Model
100% 5 sec
100% Ilsec
97,8% 3 min 40 sec 15-15 Input MBFs 99,3% 3 min 55 sec 6-6-6 Input MBFs 100% 2 min 30 sec 4,3,2,3,4,4 MBFs
96,7% 4 min 30 sec 15-15 Input MBFs 95.3% 5 min 10 sec. 6-6-6 Input MBFs 100% 3 min 15 sec 4,3,2,3,4,4 MBFs 100% 8 min 5 sec 4,3,2,3,4,4,3,5,4,3 MBFs
XXXXX
Table 1. Performance of different approaches for learning patterns
J
NEURECA has been extended by the Lin-Lee neurofuzzy algorithm and the proposed GA supported neuro-fuzzy model. The investigated problems were from one side some well-known benchmark problems (Iris, Spiral of Archimedes, XOR), where the properties of the data are well known and are used in various papers to illustrate unsupervised and supervised systems applicability. The analyzed neuro-fuzzy algorithms resulted in comparable performance, eliminating some shortcomings of the BP approach enumerated above, however sometimes with lower recognition rate for training patterns.
= _tr_a_c_e(.:....S....!:h..:-.-) trace(Sw)
where Sb is the between class and Sw is the within class scatter matrix. Through this feature selection the relative importance of measured signals in wear estimation/classification can also be evaluated. The investigations and comparisons described in this section refer to state classification of milling tools. Given 4 wear classes (sharp tools, tools with an average wear of teeth of 0.25 0.45 mm respectively, and tools with broken (missing) insert), the task was to generate and compare ANN and NF structures which are able to reliably classify unknown patterns characterizing different wear states. In first case six features (F J-F6) of the force components were selected using the SFS feature selection method [3] . 4, 3, 2, 3, 4, 4 MBFs were respectively assigned to the input linguistic variables. Corresponding to the described 4 class problem, the output linguistic variable (tool state) had 4 MBFs. Using the Lin-Lee model after initialization the, taking all the possible combinations, 4*3*2*3*4*4 = 1152 rule nodes generated. During the competitive learning phase 1138 (!) of them were deleted, resulting in a network structure with 14 rules. This self organization is a very important feature of the chosen model. The number of eliminated links was 4594. Applying genetic techniques the algorithm generally offers solutions with shorter and simpler rule sets and results in better recognition performance than the investigated previous NF approaches. Introducing empty rules gives chance to decrease the number of rules helping to evolve optimal subset of rules.
The other part of investigations referred to classification of milling tools. The cutting experiments were made in Technical University in Budapest. Carbon steel (Ck 45) was machined on a vertical milling machine with cutting speed (v = 151.6 mlmin), tooth feed (si= 0.047 mm), axial depth of cut (a = 1.5 mm) and different stages of tool wear using a six-tooth 125 mm diameter WIDAX M40 cutter with ISG Bohlerit TPAN 1603 PPN R121 inserts. Cutting force components and mechanical vibration of the workpiece holder were measured with a sampling frequency of 5 kHz. Similarly to previous ANN investigations, numerous (100 - 150) statistical and spectral features were computed from the measured signals. Because of the great number of features considered and in order to enhance the learning, estimation and classification performances of the modeling and monitoring procedures, SFS (sequential forward search) feature selection was accomplished for every group of investigations using the criterion:
555
Permission of incomplete rules offer simpler rule set. Comparing two neuro-fuzzy techniques, in genetic model the algorithm eliminated further 32 precondition links (Table 2), in contrast with traditional Lin-Lee model where the rules have full y connected in precondition site [6] . In the second case (last row of the Table I) ten best features were selected (F ]-F10) and 4, 3, 2, 3, 4, 4, 3, 5, 4, 3 MBFs were respectively assigned to the input linguistic variables. Here the number of input and output variables is too large, so the Lin-Lee neuro-fuzzy model cannot generate the large amount of fuzzy rules and regions. The elimination of this problem was the main reason of development of the proposed algorithm.
ACKNOWLEDGEMENT This work was partially supported by the National Research Foundation, Hungary , Grants No . T 014514, and TO 16512 (Fundamental research for intelligent manufacturing). A part of the work was covered by the PHARE TDQM Programme of the European Union, Grant No .: H 9305-0211 071 (REMADE).
REFERENCES [1]
[2] 1
2 3 4
5
6 7
8 9 10 11
12
13 14
Fl F2 F3 F4 FS high high -- -- high high -- -- -- med high --- -- high m high -- - high med m high m low med -- high m high m low med -- med m low -- Iow -- med m low low -- low -low low -- low -m high m low med -- med m low -- low high m low low -- low -m low -- low low low low low ---
--
F6
Outp.
--
Wearll25 Wearll25 Wear{)45 Wearll45 Wearll45 Wearll45 Broken Broken Broken Broken Sharp Broken Broken Sharp
--m high m high m high m low m low m low low
-low low low
[3]
[4]
Table 2. The set of rules remained after genetic learning
[5]
6. CONCLUSION [6]
Artificial neural networks (ANNs) are successfully applied in different fields of manufacturing, mostly where multisensor integration, robustness, realtimeness, and learning abilities are needed. Since the higher levels of the control and the monitoring hierarchy require symbolic knowledge representation and processing techniques, the integrated use of the symbolic and subsymbolic approaches is straightforward [7]. The described neuro-fuzzy systems seem to perform both the fundamental requirements of intelligent manufacturing (i.e. real-time nature, learning ability, handling of uncertainties and managing both symbolic and numerical information) and the expectation to generate sufficient rule set also for larger problems, which would be handled by usual NF models only with severe difficulties.
[7]
556
Goldberg, D. E., Genetic Algorithms in Search Optimization and Machine Learning, AddisonWesley, 1989 Egresits, Cs.; Monostori, L.: Hybrid learning approaches for intelligent manufacturing, Proc. of the 6th International DAAAM Symposium "Intelligent Manufacturing Systems", pp. 89-90, Krakow, Poland, Oct. 26-28, 1995, Monostori, L.: Hybrid AI approaches for supervision and control of manufacturing processes, Key-note paper, Proc. of the AC'95 , IV International Conference on Monitoring and Automation Supervision in Manufacturing, pp. 37-47, Poland, Aug. 28-29, 1995 Egresits, Cs.; Monostori, L.: NEURECA : An object oriented system under WINDOWS for ANN based monitoring and control of industrial processes, Collection of Summaries, 5th International DAAAM Symposium "Automation and metrology: challenge and chance", pp. 113-114. Maribor, Slovenia, Oct. 20-22, 1994 Lin, CH. Lee, C.S.G. : Neural-network-based fuzzy logic control and decision system, IEEE Trans. on Comp., Vol. 40, pp. 1320-1336, Dec. 1991 Monostori, L.; Egresits, Cs. : On hybrid learning and its application in intelligent manufacturing, Computers in Industry, Special issue on Learning in Intelligent Manufacturing Systems, 1996. (In press) Barschdorff, D., Monostori L. , Wostenktihler G.W., Egresits Cs., Kadar 8. : Approaches to coupling connectionist and expert systems in intelligent manufacturing, Computers In Industry, Special issue on Learn ing in Intelligent Manufacturing Systems, 1996. (In press).
GENETIC ALGORITHMS FOR AUTOMATIC DESIGN OF FUZZY DECISION SYSTEMS FOR INTELLIGENT MANUFACTURING
EGRESITS, CS.; MONOSTORI, L.
Computer and Automation Research Institute, Hungarian Academy ofSciences Kendeu.13-17. H-1518 Budapest, POB63, Hungary Tel: (+36 1) 1665-644, Fax: (+36 1) 1667-503, E-mail: [email protected]
Abstract: Artificial neural networks (ANNs) have been successfully applied in different fields of manufacturing. Monitoring and modelling of manufacturing processes obviously belong to the most promising areas, where real-time nature, uncertainty handling and learning abilities are essential. However, mainly because of the "black box" nature of ANNs, these solutions have limited industrial acceptance. In the paper, a combined use of the neural and fuzzy techniques in cutting tool monitoring is illustrated. We introduce a genetic algorithm based approach to overcome problems, including rule selection and redundancy checking. Finally the results are compared with ANN and previous neurofuzzy (NF) approaches.
Keywords: genetic algorithm, neuro-juzzy systems, intelligent manufacturing
1. INTRODUCTION
• to introduce a genetic algorithm based approach to overcome these problems, including: => selection of important fuzzy rules in case of large number of linguistic variables and MBFs => checking the redundancies in rule base => using fuzzy rules incomplete in precondition site to simplify and reduce rule set size • to make comparisons between the back propagation ANNs, the Lin-Lee algorithm and the suggested solution, on well known classification problems, • demonstrate the applicability of GA supported neuro-fuzzy approach for monitoring and control of manufacturing processes
Fuzzy systems are considered to be a natural link between symbolic and subsymbolic approaches. On the one hand they can work in real time circumstances and handle uncertainties as artificial neural networks, on the other hand can manage both symbolic and numerical information. However it is very hard to identify the fuzzy rules and tune membership functions (MBF) of the fuzzy reasoning. It seems that the best performance can be prospectively obtained by combining neural and fuzzy approaches, integrating their benefits. The resulting neuro-juzzy system (NF) is hybrid system, where the system architecture remain fuzzy, but using neural learning techniques, it can be trained automatically using neural and genetic learning techniques. The aims of the paper are the following: • to describe and analyse a frequently used NF model introduced by Lin-Lee [5], with special emphasis on difficulties of rule selection
2. THE LIN-LEE NEURO-FUZZY MODEL In this section, the NF model used in the investigations described later in the paper is introduced. The model basically follows the main
551
objectives of the solution described by Lin-Lee. The system consist of five layers (Figure I .). The first layer is the input layer. Each node in this layer represents a linguistic variable (F ,-F n). The second and the fourth layer contain tenn nodes which act as MBF to represent the tenns of the respective linguistic variable. The third layer is the rule node layer, where each node with its connection represents a fuzzy rule. Layerl
Layer2
Layer3
(lnP!Jf
(input Iwnn
n~)
linguistic nOdes)
_)
(
I
Layer4 (output term
nodu)
In this model, the selection of important fuzzy rules proceeds by competitive learning, which requires the generation of the whole structure at the beginning of the learning process, i.e. the net with all possible rules . That means. the third layer has as many elements, as the product of the number of different MBFs assigned to each input and output variables. This approach leads to a combinatorial explosion. This neural learning is suitable for building fuzzy logic systems with relative small number of linguistic variables. For some applications, where the number of input and output variables is large, this will create a large amount of fuzzy rules and regions which will cause learning difficulties, huge memory and hardware implementation problems. An other drawback to be mentioned is that the subset of "best" rules selected by the competitive phase is not necessarily the "best" subset of rules. The main goal of the solution described in this paper is to eliminate these shortcomings by using genetic algorithms (GA) for rule generation.
layerS (_
ilngu)stic nOdes)
~~~~~
3. GE,.'ETIC ALGORITHMS Genetic algorithms are inspired by processes that appear in biological evolution and the working of the immune system. GAs use a direct analogy of natural behaviour and work with a population of "individuals". Each individual is a potential solution of a given problem space and assigned a fitness value which measure its goodness. The fittest individuals get more opportunities to reproduce themselves. A new individual is created either by crossover or mutation operators. The fonner combines genetic infonnation of randomly chosen "parents", the latter randomly modifies parts of the infonnation describing an individual. Mutation provides a small amount of random search, and helps ensure that no point in the search space has a zero probability of getting into the population . After the randomly initialised population, over many generations, the populations evolve and could give better and better individuals [I] . Genetic algorithms are very robust due to the global searching. However, as it has been widely recognised, the GA involves intensive computation and is less efficient compared with other machine learning techniques.
Figure I . 5-layer structure of the implemented neurofuzzy model (The arrows represent signal flow in forward direction) Layer three links define the preconditions of the rules, and layer-four links incorporate the rules' consequences. Bell shaped membership functions were used in the investigations with the mean mi,j , variance fJi,j of the jth tenn of the ith input linguistic variable Fi. Perfonning precondition matching of fuzzy rules, the rule nodes in layer 3 fulfil fuzzy AND (min) operation. The nodes in layer 4 integrate the fired rules having the same consequence by a fuzzy OR (sum) operator. In forward pass layer 5 perfonns center of area defuzzification The system can be constructed from the training examples with hybrid learning schemes. The algorithm suggested by Lin and Lee consist of four consecutive steps:
4. A GA SOLUTION FOR SELECTING FUZZY RULES
• Determination of membership functions by selforganised clustering, • Selection of the most important fuzzy rules by competitive learning, • Elimination and combination of rules, • Adjustment of the parameters of the MBFs by supervised back propagation learning.
4.1 Representation A genetic algorithm can be used to discover a desirable optimal set of rules. To apply GAs, rules should be genetically coded and we need to define
552
appropriate fitness function and genetic operators. In the proposed approach instead of using a binary string, we use a integer number string (a vector) to represent a particular case or object. An individual of population is interpreted by a fuzzy rule set which is coded to a string (Figure 2).
Spreading of empty rules results in decreasing the number of rules in a rule set. Usual NF models operate well when in a rule all precondition exist, but generating rule set \\ith incomplete rules or simplifying and reducing rule set cause difficulties. In this representation introducing incomplete rules is a trivial task. We only allow "0" characters in place of any aJj . Moreover, this solution expands the search-space substantially.
R/=2 ...24 R J= 3 ... 1 1
4 ... 32 '-.t-'
4.2 Operators
~
The calculation of a new generation P(t+ J) from P(t) is done by sequentially applying the operators selection, crossover and mutation (Figure 3). The most fundamental genetic operator is the selection, in our model roulette-wheel selection was applied where the probability of selection of a specific string is proportional to its strength [I] .
R,= 4...32
Population N
Population N + I
F,,,;
Prec:oaditiolls
..
......Crossover
ConsequeDcc
""'"'""R.....,....... . . 24
~~
Rr
Mutation
r
Figure 2. Representation and genetic coding of the neuro-fuzz}' rules
Ll 1
R
Jl.r.=4 . .3 '2
Let's consider the case with n input variables (F 1Fn) and one output variable (01)' A rule has the following form: IF Fl is middle and F2 is high .. . Fn is low THEN 01 is weak, where middle, high, etc. are MBFs of the corresponding variables. In the chosen coding, the MBFs of the kth input or output variable are one by one mapped to natural numbers in interval [l ,2, ... jk.l, where jk denotes the number of all possible MBFs of the given variable. A rule set consists of maximum r pieces of rules and an individual (rule set) is coded to a string:
Figure 3. Calculation of new generation in applied Genetic algorithm Crosso\'er regarded as the most powerful genetic operator and the main engine for exploration in GAs. This is in touch \\ith the building block hypothesis, as it is the operator responsible for shuffling and recombination of building blocks. R,=!..14 R,=2.,.ll R,-2..22
R,- !...l J R,=: 11 R J ::;:; ; : ;
. . . . . . ...
where a/i and bl are the mapped values of the corresponding MBFs of the ith input and the output variable respectively, in Ith rule. For example if the FI input variable has four membership functions very_high, high, low and very_low, and FI is high in the ith rule then ai ]'=2. A population contains strings 'with the same length. A special rule named "empty" rule is introduced in the representation which basn't got any preconditions and consequences. Every rule has the same length, therefore the empty rule is coded with n+ I (number of inputs + number of outputs ) "0" characters.
R. 1=3 J 3
Parent 1
R, -1 .,3 ~
R...,-3 .. .33 R, =1...34
Parent 2
Child 1
Cbild 2
Figure 4. Single point crossover The most simple crossover takes two selected individuals and cuts their chromosome strings at a
553
randomly chosen position only at the bounds of rules producing two "heads" and two "tails". The two offsprings come into existence from parents changing their tails (Figure 3). In the initial stage to get more genetic mix ing we used uniform crossover because the probability of finding building blocks is munch lower and the convergence is quicker (Figure 4). Uniform crossover radically different to one point crossover. Each gene in the offspring is created by copying the corresponding gene from one or the other parent chosen according to a randomly generated crossover mask. Crossover isn't applied to all pairs, usual probability of crossover is 0.6-0.8. After every crossover the redundant rules in a rule set are replaced by an empty rule, so no more than one copy remains from a given type of rule .
We investigated effects of the GAs parameters and comes to the conclusion that as a rule of thumb, the crossover rate or probability should be about 0.6, and mutation rate should be about 0.005 . The system is not sensitive to small changes to crossover rate and using 0.005 gives good learning performance even when crossover rate is zero. To preserve the best individual of the nth generation from being changed by the genetic operators, this individual might be copied to the next generation. This strategy called fittest survive does not always increased the performance of the algorithm. 4.3 Fitness function
In this case the fitness function is a weighted quadratic error of the rule set counted as the sum of quadratic errors of the training patterns multiplied by recognition rate of the system . This weight is related to the number of non-empty rules of rule set and assists spreading of empty rules for reduce of rule set. For fitness calculation the system has to generate the rules for the net and run the classification process.
Crossover mask
R,· 1... 1 •
R, . ] . 14
. . . ..
~L,u,
R:·2: J I R,·2 .22:
..... ~
R _¥;fJ R.· I ,.l4 Parent 2
Parent 1
Child 1
Child 2
5. EXPERIMENTS Figure 5. Multi point uniform crossover
We developed in the Computer and Automation Research Institute, Budapest an ANN and neurofuzzy simulator caIled NEURECA . It was written in C++ using its object oriented nature enabling to dynamicaIly vary the network structure during learning and to implement different ANN models including various neuro-juzzy approaches. NEURECA provides the following main features in an integrated framework: • definition of different statistical and spectral features for various channels • on-line feature computation, • automatic feature selection, • manipulation , visualization of pattern files . • ANN learning with back propagation (BP) algorithm , • different neuro fuzzy learning techniques including whole Lin-Lee model • classification, estimation of unknown patterns, • standardized (DOE) interfaces to other programs, etc. [4] The back propagation (BP) part of the system was successfuIly applied for different problems (e.g. first column of the Table 1). However, ANN models have a set of problems: • lengthy training times. • dependence on the initial parameters, • lack of a problem-independent way to choose appropriate network topology, • incomprehensive (black box) nature of A1'-.'Ns .
Individuals become very similar before a nearly optimal solution is reached, thus preventing any further progress. Mutation acts against this, by constantly generating new rules, thus prevent the population from getting trapped in a local maximum in the search space. However, the action of mutation can sometimes result in the loss of good individuals, so the need to prevent convergence has to be balanced against the inevitable loss of efficiency due to the disruption of good genetic material. Mutation is applied to each child individuaIly after crossover. It randomly modifies each character of the code with a small probability (Figure 6).
R20/d= ..
R I=2 ...2 I Rl=2 ...22
,J
2 ... 22
~. 1 2 3
~
R.= 2 .. 3 I
~
=
Candidates
R 2new= 2 ... 3 2 •. ...
random selection
R I = 2.2 I R1= 2... 3 2
R.= 2 .. 3 I
Figure 6. Mutation
554
XOR with 3 input
Two spiral
Iris
ANN 8ackpropagation 100% 25 sec 350 iteration 100% 4,5 hours 95000 iteration 100% 2,5 hours 68000 iteration
Milling tool classification with 6 inputs
100% 2 min 15 sec
Milling tool classification with 10 inputs
100% 5 min 30 sec
Lin-Lee Neuro-Fuzzy Model
GA supported Neuro-Fuzzy Model
100% 5 sec
100% Ilsec
97,8% 3 min 40 sec 15-15 Input MBFs 99,3% 3 min 55 sec 6-6-6 Input MBFs 100% 2 min 30 sec 4,3,2,3,4,4 MBFs
96,7% 4 min 30 sec 15-15 Input MBFs 95.3% 5 min 10 sec. 6-6-6 Input MBFs 100% 3 min 15 sec 4,3,2,3,4,4 MBFs 100% 8 min 5 sec 4,3,2,3,4,4,3,5,4,3 MBFs
XXXXX
Table 1. Performance of different approaches for learning patterns
J
NEURECA has been extended by the Lin-Lee neurofuzzy algorithm and the proposed GA supported neuro-fuzzy model. The investigated problems were from one side some well-known benchmark problems (Iris, Spiral of Archimedes, XOR), where the properties of the data are well known and are used in various papers to illustrate unsupervised and supervised systems applicability. The analyzed neuro-fuzzy algorithms resulted in comparable performance, eliminating some shortcomings of the BP approach enumerated above, however sometimes with lower recognition rate for training patterns.
= _tr_a_c_e(.:....S....!:h..:-.-) trace(Sw)
where Sb is the between class and Sw is the within class scatter matrix. Through this feature selection the relative importance of measured signals in wear estimation/classification can also be evaluated. The investigations and comparisons described in this section refer to state classification of milling tools. Given 4 wear classes (sharp tools, tools with an average wear of teeth of 0.25 0.45 mm respectively, and tools with broken (missing) insert), the task was to generate and compare ANN and NF structures which are able to reliably classify unknown patterns characterizing different wear states. In first case six features (F J-F6) of the force components were selected using the SFS feature selection method [3] . 4, 3, 2, 3, 4, 4 MBFs were respectively assigned to the input linguistic variables. Corresponding to the described 4 class problem, the output linguistic variable (tool state) had 4 MBFs. Using the Lin-Lee model after initialization the, taking all the possible combinations, 4*3*2*3*4*4 = 1152 rule nodes generated. During the competitive learning phase 1138 (!) of them were deleted, resulting in a network structure with 14 rules. This self organization is a very important feature of the chosen model. The number of eliminated links was 4594. Applying genetic techniques the algorithm generally offers solutions with shorter and simpler rule sets and results in better recognition performance than the investigated previous NF approaches. Introducing empty rules gives chance to decrease the number of rules helping to evolve optimal subset of rules.
The other part of investigations referred to classification of milling tools. The cutting experiments were made in Technical University in Budapest. Carbon steel (Ck 45) was machined on a vertical milling machine with cutting speed (v = 151.6 mlmin), tooth feed (si= 0.047 mm), axial depth of cut (a = 1.5 mm) and different stages of tool wear using a six-tooth 125 mm diameter WIDAX M40 cutter with ISG Bohlerit TPAN 1603 PPN R121 inserts. Cutting force components and mechanical vibration of the workpiece holder were measured with a sampling frequency of 5 kHz. Similarly to previous ANN investigations, numerous (100 - 150) statistical and spectral features were computed from the measured signals. Because of the great number of features considered and in order to enhance the learning, estimation and classification performances of the modeling and monitoring procedures, SFS (sequential forward search) feature selection was accomplished for every group of investigations using the criterion:
555
Permission of incomplete rules offer simpler rule set. Comparing two neuro-fuzzy techniques, in genetic model the algorithm eliminated further 32 precondition links (Table 2), in contrast with traditional Lin-Lee model where the rules have full y connected in precondition site [6] . In the second case (last row of the Table I) ten best features were selected (F ]-F10) and 4, 3, 2, 3, 4, 4, 3, 5, 4, 3 MBFs were respectively assigned to the input linguistic variables. Here the number of input and output variables is too large, so the Lin-Lee neuro-fuzzy model cannot generate the large amount of fuzzy rules and regions. The elimination of this problem was the main reason of development of the proposed algorithm.
ACKNOWLEDGEMENT This work was partially supported by the National Research Foundation, Hungary , Grants No . T 014514, and TO 16512 (Fundamental research for intelligent manufacturing). A part of the work was covered by the PHARE TDQM Programme of the European Union, Grant No .: H 9305-0211 071 (REMADE).
REFERENCES [1]
[2] 1
2 3 4
5
6 7
8 9 10 11
12
13 14
Fl F2 F3 F4 FS high high -- -- high high -- -- -- med high --- -- high m high -- - high med m high m low med -- high m high m low med -- med m low -- Iow -- med m low low -- low -low low -- low -m high m low med -- med m low -- low high m low low -- low -m low -- low low low low low ---
--
F6
Outp.
--
Wearll25 Wearll25 Wear{)45 Wearll45 Wearll45 Wearll45 Broken Broken Broken Broken Sharp Broken Broken Sharp
--m high m high m high m low m low m low low
-low low low
[3]
[4]
Table 2. The set of rules remained after genetic learning
[5]
6. CONCLUSION [6]
Artificial neural networks (ANNs) are successfully applied in different fields of manufacturing, mostly where multisensor integration, robustness, realtimeness, and learning abilities are needed. Since the higher levels of the control and the monitoring hierarchy require symbolic knowledge representation and processing techniques, the integrated use of the symbolic and subsymbolic approaches is straightforward [7]. The described neuro-fuzzy systems seem to perform both the fundamental requirements of intelligent manufacturing (i.e. real-time nature, learning ability, handling of uncertainties and managing both symbolic and numerical information) and the expectation to generate sufficient rule set also for larger problems, which would be handled by usual NF models only with severe difficulties.
[7]
556
Goldberg, D. E., Genetic Algorithms in Search Optimization and Machine Learning, AddisonWesley, 1989 Egresits, Cs.; Monostori, L.: Hybrid learning approaches for intelligent manufacturing, Proc. of the 6th International DAAAM Symposium "Intelligent Manufacturing Systems", pp. 89-90, Krakow, Poland, Oct. 26-28, 1995, Monostori, L.: Hybrid AI approaches for supervision and control of manufacturing processes, Key-note paper, Proc. of the AC'95 , IV International Conference on Monitoring and Automation Supervision in Manufacturing, pp. 37-47, Poland, Aug. 28-29, 1995 Egresits, Cs.; Monostori, L.: NEURECA : An object oriented system under WINDOWS for ANN based monitoring and control of industrial processes, Collection of Summaries, 5th International DAAAM Symposium "Automation and metrology: challenge and chance", pp. 113-114. Maribor, Slovenia, Oct. 20-22, 1994 Lin, CH. Lee, C.S.G. : Neural-network-based fuzzy logic control and decision system, IEEE Trans. on Comp., Vol. 40, pp. 1320-1336, Dec. 1991 Monostori, L.; Egresits, Cs. : On hybrid learning and its application in intelligent manufacturing, Computers in Industry, Special issue on Learning in Intelligent Manufacturing Systems, 1996. (In press) Barschdorff, D., Monostori L. , Wostenktihler G.W., Egresits Cs., Kadar 8. : Approaches to coupling connectionist and expert systems in intelligent manufacturing, Computers In Industry, Special issue on Learn ing in Intelligent Manufacturing Systems, 1996. (In press).