MATHEMATICAL COMPUTJZR MODELLING
PERGAMON
Mathematical
and Computer
Modelling
29 (1999) 71-81
Fuzzy Neural Network Modeling for Product Development CHI-HSING Hsu, B. C. JIANG* AND E. S. LEE Department of Industrial Engineering Yuan-Ze University, Chung-Li, 32026, Taiwan, R.O.C. (Received and accepted December 1998)
Abstract-A fairly general product development model is formulated and analyzed based on multiple attribute decision making with emphasis on the treatment of the linguistic and vague aspects by fuzzy logic and up-dating or learning by neural network. Due to the representative ability of fuzzy set theory and the learning or intelligent ability of neural network, the proposed approaches appear to be an effective tool for handling vague and not well-defined systems. @ 1999 Elsevier Science Ltd. All rights reserved. Keywords-Product ing.
development,
Neural network, Fuzzy logic, Multiple attribute decision mak-
INTRODUCTION Too many factors, most of them vague and not well defined, need to be considered for new product development. However, for a given product with known characteristics and environment, these factors frequently can be considerably reduced. Thus, it is difficult to formulate a general modeling and decision approach, which can be used for any product. However, with the recent advances in decision making and the development in intelligent algorithms such as fuzzy set theory and neural network, a general modeling and decision approach can be developed with emphasis on the up-dating or learning aspects to compensate for the vagueness of the problem. In this paper, a general decision model for new product development is proposed based on multiple attribute decision making and by the combined use of fuzzy set theory and neural network. Although only a simplified model is considered to illustrate the approach, it can be generated easily by adding other important influential factors. After formulating the general product development problem, different approaches of neural network modeling with the assistance of fuzzy if-then rules are investigated and compared based on this newly formulated problem.
THE
GENERAL
PRODUCT
DEVELOPMENT
PROBLEM
New product development is a complex and, most importantly, not well-defined problem. There exist many factors, which influence the success of a new product. However, in order to simplify *Author to whom all correspondence should be addressed. The authors gratefully acknowledge financial support from the National China, under project NSC 86-2623-D-155-005 and NSC 87-2213-E155-024. 0895-7177/99/$
Science Council,
- see front matter. @ 1999 Elsevier Science Ltd. All rights reserved.
PII: SO8957177(99)00082-5
Taiwan,
Typeset
Republic
by J++TEX
of
C.-H.
72
1 Figure
1
1
1
1. Hierarchical
0
A
vc
0 Figure
Hsu
et al.
2
1
representation
1
3
of product
0.75
0.25
0.5
c
M
H
0.25
0.5
0.75
2. Membership
functions
1 . . . ..I
N
1
development.
1.0
Market Potential
VH
1.0
Time
of attributes.
the problem and also to illustrate the approach, only the following three most important factors or attributes are considered in this investigation: (1) market potential, (2) product development cost, and (3) time needed to reach actual on-line production. All these three attributes are fairly vague and fuzzy and can be represented linguistically [1,2]. This is especially true for market potential, which can only be represented linguistically. For example, the survey results may be expressed as the market potential is high for this new product. Thus, fuzzy sets and fuzzy logic are ideally suited for representing these attributes. These attributes form the input of the model and the outputs are the criteria or desirability of the proposed product development approach. Figure 1 illustrates the hierarchical linguistic repre-
Fuzzy Neural Network Modeling
73
Table 1. Fuzzy if-then rules.
16 17
H
I
H
18
H
19
M
1
20
1
M
I
21
I
M
t
I
M /
C
S 1
M vc
M
B4
1
B5
vs
Cl
I vc I s I C I vs I
/ I
B4
M
I
I
c2 c2
22
M
C
S
c3
23
M
VC
M
c3
24
M
M
vs
c3
25
M
M
S
c4
26
M
C
M
c4
27
M
M
M
c5
I II
sentation, where the goal or the desirability is the linguistic variable and the attributes and the alternatives form the values of the linguistic variable, which also known as linguistic terms. Since these representations are fuzzy and approximate, neural network learning is a necessity to up-date or improve this approximate model. The alternatives or the different grades of the attributes are represented linguistically in Figure 2. Each attribute is divided into several different grades based on a universe of discourse of [O,l]. Each grade is represented by a triangular membership function. The attribute market potential has the linguistic grades: VL (Very Low), L (Low), M (Medium), H (High), VH (Very High). The attribute cost has the linguistic levels: VC (Very Cheap), C (Cheap), M (Medium), H (High), VH (Very High). The attribute development time has the linguistic levels: VS (Very Short), S (Short), M (Medium), L (Long), VL (Very Long). The problem is to choose the alternatives so that the combined values of the three attributes form the desired best goal in a certain sense. Notice that this is a multiple criteria problem and, in general, the three criteria cannot mutually compensate. For example, one product may have the best market potential (VH) but has a highly uncertain high development cost (H) and the other only has a medium market potential but has a fairly certain medium cost (M). The
C.-H. HSU et
74
CLASS
al.
C5 BS(C4) B4(C3) B3(C2) AS(B2) A4(Bl) A3
I
A2
Al
Figure 3. Membership function for output ranking score.
decision maker must choose between these two based on his judgement of the environment and other factors. Thus, the best goal cannot be definitely defined except based on the characteristics and the environment. Based on the above discussion, the output variable can also be divided into different fuzzy levels. The connections between the input attributes and the output goals can be made by the use of fuzzy rules. Table 1 illustrates these fuzzy if-then rules [2-51. For example, for Criteria 2, we have the following. If the market potential is VH and the cost is VC and the time needed is VS then the grade of the output or goal is Al. The expression if-and-then is in fuzzy logic terms. To simplify the illustration, only three levels of each attribute with a total of 27 rules are considered in Table 1. The different output grades for these 27 rules are shown in Figure 3. The output ranking has three classes designated as A, B, and C, and each class has five grades. For example, class A is divided into: Al, A2, A3, A4, and A5. The fuzzy rules or the interconnections between the input and the output can first formulated approximately based on the vague knowledge of the actual system. These approximate interconnections can be refined for any given system by actual learning using neural network. The training sample sets for the neural network can be obtained from the actual system experts. To investigate the effectiveness of the approach, several neural network learning models will be considered in the following.
COMPETITIVE
LEARNING
NEURAL
NETWORK
The Kohonen competitive learning network [3,4,6,7], or winner-take-all unsupervised learning, is first used to investigate the product development problem. The learning is based on clustering similar input data into the same group and dissimilar ones to different groups.
The network
architecture is shown in Figure 4. The learning rule involves the following two-stage computation: (1) similarity matching
I/-m(k) II= 5
min l
{II 2
_
,j(“)
II> ,
(2) updating Wk by q(k+l) @+l)
= m(k) + a(W
= q(k),
forj=1,2
(x-m&k)) )
,...,
7L, j#l,
where ock) is a suitable learning constant at the kth time step and w is the normalized weight vector. The above learning process stops when two consecutive weight changes become smaller than a prespecified small number. Figure 5 illustrates the different clusters based on the 27 rules in Table 1. There are 27 cluster blocks in this three-dimensional figure. Each axis is divided into three segments or three clusters
75
Fuzzy Neural Network Modeling
Cluster 1 . ..Cluster i . .. Cluster n (Cluster)
t
t
t
output layer
input layer: x
Xl Figure 4. Kohonen
X2
X3
winner-take-all
1
Figure 5. The cluster
network.
X
Market Potential
learning space.
boundaries: 0, l/3, 213, and 1. The centroids of the three segments or cluster blocks are 0.25, 0.5, and 0.75. For example, the coordinate of the cluster center for block 1 is (0,75,0.75,0.75), and for block 2 is (0.75,0.75,0.5). Based on these clusters, Table 2 is obtained. The 27 clusters are mapped into nine ranks. The inputs, or the if part, are represented by the three attributes each of which is divided into three segments. The output, or the then part, is represented by the 27 clusters, which are mapped into nine ranks as shown in Table 2.
with
The learning and testing phases were carried out using the MATLAB
software.
The initial
weights were obtained randomly. The training sample was generated using random numbers. The output target has 27 clusters that mapped into nine ranking classes. Thus, there are 27 nodes in the competitive layer. The training parameters are: (1) 1500 sets of random numbers are used to generate the inputs for the three attributes, (2) initial learning rate is 0.05, (3) 20000 total training epochs were carried out. The test data are generated by random numbers and are summarized in Table 3. To reduce the number of alternatives and, at the same time, to be able to test the trained network effectively, only two of the most desirable alternatives for a given attribute are considered. These two alternatives are VH and H for market potential, VC and C for cost, and VS and S for time needed. Eight alternatives were generated by the use of random numbers. The aggregate values are obtained based on Figure 5, where the axes of x, y, and t represent market potential, cost and time needed, respectively. For example, for Criteria 1 for market potential, from Figure 5, we have 2 = 0.75VH + 0.5H = 0.75(0.9) + 0.5(0.1) = 0.725. Using the trained network and the just generated test data, the last column in Table 3 is obtained. Only ranking classes 1, 2, 4,
76
C.-H. HSU et al. Table 2. Fuzzy look-up table for competitive
learning.
Table 3. The ranking of eight alternatives by competitive
learning method.
Fuzzy Neural Network Modeling
77
5, and 6 are represented. Compared to the desired class column, the result is not very good. It appears this approach cannot distinguish between boundary points for adjacent sample blocks. Thus, this approach is suitable for a preliminary screening only.
LEARNING (LVQ)
VECTOR NEURAL
QUANTIZATION NETWORK
To improve the above approach, learning vector quantization [7], which has a limited supervised learning, is investigated next. The LVQ technique is due to Kohonen and is an adaptive data classification method based on training the data with known desired class information. Although a supervised training method, LVQ employs unsupervised competitive learning data clustering technique to preprocess the data set and to obtain the cluster centers. LVQ’s network architecture is essentially the same as that shown in Figure 4 except that each output unit has a known class that it represents. The LVQ learning algorithm involves two steps. In the first step, an unsupervised learning data clustering method is used to locate several cluster centers without using the class information. In the second step, the class information is used to fine-tune the cluster centers to minimize the number of misclassified cases. The training algorithm is based on whether the current input belongs to the same class or not. Thus, we have the following. If x and Wk belong to the same class, update the weight Wk by Awk = n(x - wk). Otherwise, update Wk by A?_!&=
-q(x
-
wk).
The learning rate n is a positive small constant and decreases with the number of iteration. The input vector and the given target classes are obtained from Table 1 and Figure 3. The fuzzy if-then rule is converted to training sets as that listed in Table 2. Using the MATLAB software, the training and the test phases are carried out. The 27 training samples constitute the input and the output has nine ranking classes. The network was trained by three separate experiments with initial learning rates 0.05, 0.01, and 0.08 and with 5000, 3000, 10000 epochs, respectively. The test phase can use any of the 27 input training samples. To test the effectiveness of the trained network, the input data listed in Table 3 are again used. The test results are listed in the last column of Table 4. Compare to the competitive network, the results improved. However, the classification is still not perfect and it cannot separate alternatives 2, 4, and 7. Table 4. The ranking of eight alternatives by LVQ method.
1 Alternative
BACK
PROPAGATION
NEURAL
NETWORK
Instead of using the various combinations of the neuro-fuzzy network, the backpropagation network algorithm is used directly to investigate the product development problem. To obtain
C.-H.
HSU et al.
Table 5. Fuzzy if-then rules for back propagation.
Fuzzy Neural Network Modeling
79
Table 5. (cont.)
I
54
I
0.7
I
I
I
0.3
0
0.9
0.1
I
0.0
I
I
0.7
0.3
I
0
0.913889
Table 6. The ranking of eight alternatives by BPNN method.
the training samples, the Center Of the Area (COA) defuzzification method is used. The “and” in the “if’ part is treated as conjunction and with triangular fuzzy numbers, the COA defuzzification equation for discrete system can be represented by
where n is the number of quantization levels, and ps(fj) is the membership at the quantized level j. Using the above equation and the fuzzy if-then rules in Table 1 with the help of Figure 3, the 54 training samples listed in Table 5 are obtained. The backpropagation network is constructed in three layers: nine nodes in input layer, five nodes hidden layer, and one node in the output layer. The network is trained by using the usual gradient method. The equations for weight adjustment are
A~2[4lil(t) = vmq + aAw2[4[.A(t - 11, Aw,[il[.il(t) = my& +~Awl[4[A(t - 11,
80
C.-H. HSIJ et al.
where ej: the errors in output at the output layer, tj:
the errors in output at the hidden layer,
70: the learning rate parameter for the output layer, qh: the learning rate parameter for the hidden layer, (u: momentum parameter. The parameters used are: q = 0.9, a = 0.7, with iteration number 30000. The system error is limited to 5 x 10m6. The convergence behavior for this training experiment is shown in Figure 6. E*10-6
t
20
15
10 5
I
I >
2.5
5
Figure 6. Convergence
7.5
Iteration number* 10 3
behavior.
Using the trained network, the test or the recall phase is carried out by using the same input data as that shown in Table 3. The ranks obtained for the eight alternatives are listed in the last column of Table 6. The ranking order are expressed as: Al > A4 > A7 > A6 > A2 > A8 > A3 > A5. The resulting ranking order is not consistent to the desired outputs for A2 and A8. This discrepancy may be caused by the fact that numerical values are too near to be distinguished.
DISCUSSIONS New product development is a typically vague and not well-defined process, and thus, is ideally suited for fuzzy and intelligent modeling. To illustrate the use of these intelligent systems, a general new product development model is formulated and investigated by the use of fuzzy ifthen rules and neural network learning. Although the model is a simplified version, it can be generated easily to model a more complicated system. The basic idea is that modeling by first using fuzzy approximate linguistic if-then rule representation and then fine tuning or improving the accuracy by neural network learning. Obviously, many different neuro-fuzzy network [1,8] can also be used to model the new product development process. For example, fuzzy adaptive network [3,7,8], which combines the fuzzy representation with neural network learning, appears to be an ideal approach for new product development.
REFERENCES 1. G.J.
Klir and T.A. Folger, Fuzzy Sets, Uncertainty, and Znfomation, Prentice-Hall, Englewood Cliffs, NJ, (1992). 2. H.J. Zimmermann, fizzy Set Theory and Its Applications, Kluwer, Boston, MA, (1991). 3. J.-f&R. Jang, C.T. Sun and E. Mizutani, Neuro-Fuzzy and Soft Computing, Prentice-Hall, Englewood Cliffs, NJ, (1997). 4. L.X. Wang, Adaptive Fuzzy Systems and Control, Prentice-Hall, Englewood Cliffs, NJ, (1994).
Fuzzy Neural Network Modeling
81
5. RR. Yager and L.A. Zadeh, Editors, An Introduction to Fuzzy Logic Applications in Intelligent Systems, Kluwer, Boston, MA, (1992). 6. CT. Lin and C.S.G. Lee, Neural Fuzzy Systems, Prentice-Hall, Englewood Cliffs, NJ, (1996). 7. V.B. Rao and H.V. Rae, C++ Neural Networks d Filtzy Logic, MIS Press, New York, (1995). 8. C.B. Cheng and E.S. Lee, Applying fuzzy adaptive network to fuzzy regression analysis, Computers Math. Applic. (to appear).