- Email: [email protected]
A fuzzy CMAC model for color reproduction
WI~--~-,,--L~ -~,;rlrt, mu
FUZilY
sets and systems Fuzzy Sets and Systems91 (1997) 53-68
ELSEVIER
A fuzzy CMAC model for color reproduction Jar-Shone
K e r a, C h a o - C h i h
H s u b, Y a u - H w a n g
K u o b'*, B i n - D a L i u a
"Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan, ROC blnstitute of Information Engineering, National Cheng Kung University, Tainan, Taiwan, ROC
Received September 1995;revised February 1996
Abstract
Color reproduction is a complex nonlinear mapping problem due to gamut mismatch, resolution conversion, quantization, nonlinear color relationship between scanner and printer. To solve such a complex problem in color reproduction, this paper proposes a fuzzy CMAC model, which adopts a special parallel fuzzy inference-like process to realize the function similar to higher-order CMAC. In this model, recursive B-spline receptive field functions are replaced by fuzzy sets with bell-shaped membership function, and the weights to evaluate output values are also not crisp values but fuzzy sets. The learning algorithm is based on the maximum gradient method. For the situations of insufficiently or irregularly distributed training patterns, this paper develops a sampling method to generate uniformly distributed training patterns. According to experimental results, the proposed fuzzy CMAC model has shown its effectiveness on color reproduction and general function approximations. Besides, it has advantages of fast learning speed, simple computation, and high stability on model parameters. © 1997 Elsevier Science B.V. Keywords: Fuzzy logic; Cluster analysis; CMAC; Color calibration
1. Introduction
Due to the rapid progress on the technologies of computer and color scanning/printing devices, the extraction, manipulation, and reproduction of color documents have become interesting research topics. One of the major problems among these topics is how to reproduce the original color information of documents or images when they are scanned in and then printed out. Unfortunately, the
*Fax : 886 6 274 7076; e-mail: [email protected].
color mismatch phenomenon is very severe and undesired in the existing color reproduction systems [10, 21]. The color mismatch may result from the nonlinear characteristics, for example, gamut mismatch, resolution conversion and quantization, between the scanning devices and the printing devices [19, 23]. The principal purpose of color reproduction is to maintain color resemblance between the reproduced and original images, and thus it is a problem of nonlinear function approximation. Conventionally, most color calibration systems are based on a table look-up scheme, where the content of the look-up table is manually defined. Such an approach requires huge design effort and lacks self-adaptivity.
0165-0114/97/$17.00 © 1997 ElsevierScienceB.V. All rights reserved PII S0165-01 14(96)00083-8
54
,Z-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53 68
To develop a color calibration model with selflearning and self-adaptivity capabilities, Chang [7] and Kang [10] proposed, respectively, an approach that uses multilayer feedforward network with backpropagation learning algorithm to calibrate color mismatch. However, their methods may have some drawbacks such as low learning speed and high sensitivity to the order of presentation of the training data. Unlike the global weight updating scheme of backpropagation neural networks, Cerebellar Model Articulation Controller (CMAC) model, proposed by Albus [1-5], is characterized by a local weight updating scheme which exhibits the advantages of fast learning and high convergence rate. The higher-order CMAC with B-spline receptive field functions [22] can further improve the accuracy of function approximation. CMAC has been successfully applied in many fields such as robot control [-16], pattern recognition [15], and signal processing [12]. Recently, it is also adopted for color calibration [11]. The operation of CMAC model can be considered as a mechanism whose output is obtained by adding a fixed number of weights. The input vector is transformed into an Na-dimensional association vector in which each element is associated with an individual weight cell. There are always N~. nonzero elements in the association vector so that only NL crisp weights contribute to the evaluation of an output value. There is one significant criterion in determining which weight cells are to be activated for a given input vector. The number of overlapped weights for two input vectors is inversely proportional to their distance (e.g., the hamming distance). If the distance exceeds a predefined value, there will be no overlapped weight between the input-correlated association vectors. Such a feature is the well-known local generalization characteristic of CMAC-based models. But there exists some problems when one CMAC-based model is applied. First, CMAC will be computationally complicated when the order of B-spline receptive field functions is high. Second, the convergence speed of CMAC is more or less unstable as the model parameters alter, which will increase the design effort in choosing a set of appropriate parameters. Therefore. this paper intends to
develop a new model for color calibration, which can overcome the mentioned problems. Embedding fuzzy inference capability into conventional CMAC model may be a good alternative to reach the goal of this paper. Thus, a novel fuzzy CMAC model, called FCMAC, is proposed in this paper to fully exploit the advantages of fuzzy set theory and the local generalization feature of CMAC model. To reserve the characteristics of CMAC, FCMAC performs a special parallel fuzzy inference process which has the following features: (1) the number of fuzzy rules fired at each cycle to get an output value is fixed; (2) the decision of the fuzzy rules to be fired and their firing strength is realized by the mapping from input vector to association vector; (3) the evaluation of output values corresponds to three operations: parallel rule firing, conflict resolution, and defuzzification. Besides, this paper also proposes a training pattern sampling algorithm, based on fuzzy clustering, similarity measure, and interpolation approaches, to generate a set of uniformly distributed training patterns from raw training data. This sampling algorithm is helpful in improving the accuracy of function approximation with FCMAC, especially when the raw training data are insufficient or irregularly distributed. This paper is organized as follows. Section 2 presents the FCMAC model. Section 3 explains the sampling algorithm for generating a set of uniformly distributed training patterns. Section 4 discusses the experimental results. Finally, some conclusions are given in Section 5.
2. Fuzzy CMAC model Fig. 1 shows the structure of FCMAC. Similar to conventional CMAC model, FCMAC approximates a nonlinear function y = f ( x ) by using two primary mappings: 4s:X ~ A,
(1)
O: A ~ D,
(2)
where X is a continuous s-dimensional input space, A is an NA-dimensional association space, and D is
55
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
fuzzy weights
I
,!
Desired actuatorsignal ~x
Adju~ activeweights(~
/
~
~
Nonzero Excitation
.... X : Input
A : Association
Actuator
•
Zero Excitation
D : Defuzzification
Fig. 1. Structure of FCMAC model.
a one-dimensional output space. But these two mappings are realized by fuzzy operations unlike conventional CMAC models. The function ~k(x) is usually fixed and maps each point, x, in the input space X onto an association vector ~ e A that always has NL nonzero elements. Formally, we can write it as = ~(x)
= (~t,~2 ....
,~j,
. . . , ~N,),
where ~i is derived from the composition of fuzzy receptive field functions and sensory inputs, and 0 ~< ~j <~ 1 for all components in ~. The second mapping, 0(~), computes a scalar output. In fact, it is a special mapping which executes a parallel fuzzy inference-like procedure. In this procedure, each nonzero association cell ~j retrieves a corresponding weight cell wj which is a fuzzy number. Moreover, each (~j, wj) pair constitutes the inference channel of a fuzzy rule, where ~j is the matching degree of the condition part while wj is the conclusion part of that rule. In other words, there are always NL(~j, wi) fuzzy rules activated in each FCMACs recall cycle to get final output. The mapping of 0(~), that is, the parallel fuzzy inference procedure, can be interpreted by employ-
ing the truth-value flow model 1-24], as shown in Fig. 2. First, the sensory input vector x is used to decide the addresses of nonzero association cells and calculate the values of those nonzero association cells. The addresses are used to retrieve Nt corresponding fuzzy weights. Then each fuzzy weight is inferred to produce a partial fuzzy output by applying the value of its corresponding association cell as input matching degree (rule firing strength). Third, all NL partial fuzzy outputs are aggregated to a global fuzzy conclusion by a certain conflict resolution strategy. Finally, the global fuzzy conclusion is defuzzified into a scalar output by the centroid of area (COA) approach. FCMAC model is basically a special table lookup model in which fuzzy weights are stored and manipulated by means of fuzzy inference. It does not operate by mathematically analyzing the dynamics of the controlled plant and then solving equations, but computes the control amount by referring to a fuzzy weight memory table. The weight memory table is established by storing the mappings between input values and control amounts. FCMAC learns the control function over the input space by modifying or generalizing its internal data distribution obtained from previous
56
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
Matching degree calculation
Partial fuzzy outputs Conflict resolution
M = vM
---'q
Defuzzification
i
. IM,(y) o, I
('F Fig. 2. The truth-value flow model of fuzzy inference.
learning. Once the fuzzy weight memory table is built, the control amount is then obtained by executing the two mappings represented by Eqs. (1) and (2).
2.1. Fuzzy receptive field functions It is well-known that many biological sensorimotor control structures in the brain are organized in a way that neurons possess locally tuned overlapping receptive fields [9,20]. Several computational paradigms of neural network, such as CMAC and radial basis functions (RBF) [17, 18], have been quite successful in representing complex nonlinear functions by using similar organization principles. These networks construct local approximation to multi-input/multi-output functions, which is similar to the approximation scheme of using piecewise polynomials or other basis functions on a set of training data. In FCMAC model, we adopt normal bell-shaped membership functions as the receptive field functions:
Before further discussing, some terms used in this paper must be declared: i ( = 1..... u), the index of input dimension being considered; x (=(Xl, x2 ..... xi ..... xu)), input vector; X .... ~, the upper bound of dimension i; Xmin.i, the lower bound of dimension i; Rg, the resolution of dimension i; Di, the width of quantization interval on dimension i; r, the effective radius of the receptive field function by using Di as unit; and q~(x), the quantized position of input vector x on dimension i. The quantized position of input vector x on dimension i can be determined by the following equation:
qi(x) = [ x l
Di
where L" J denotes the mathematical floor function. Therefore, when the bounded input space and the width of quantization interval are given, the quantized resolution on dimension i can be computed as
/~(x) = e -((x-m)/~)~, where m represents the mean value and ~ represents the variance.
_1
....
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
57
u i 1
....
I! X
m in,i':
S
m iI
m
i J
m
m ux,i
i q
Fig. 3. Fuzzy receptivefield functions. 1.00
_
P
0.00 i
mj
i mj+l
Fig. 4. Decision of the receptive field variance. Now, we derive the bell-shaped receptive field function in Eq. (3): #}(x) = e -((x'-m))/')2,
(3)
where m} denotes the jth mean on dimension i, which can be expressed by the following formula: mji = X m i n , i + (j _ ½) x Di
where j E I-- r + 1, Ri + r].
(4)
The receptive field function #~(x) denotes the grade of x belonging to interval [-2j_ i 1,2j)i where /~j-- 1 = (qi(x) - 1) × Di and 2~ = qi(x) x Di. When xi is in the middle point of that interval, it will have the full membership of p~(x). Fig. 3 illustrates an example of the receptive field functions of F C M A C model.
The decision of the variance tr of #~(x) is based on the idea that allocates any two adjacent fuzzy receptive field functions to have a proper overlapping region. Assume that by this arrangement the two adjacent receptive field functions will intersect at the point whose membership grade is fl ( ~ (0, 1]), as illustrated in Fig. 4. Then a, expressed in Eq. (6), can be easily derived from Eq. (5): fl = e x p ( _
( ((ml + m 2 - ) a / 2 ) - - m l ) 2 ) ,
/((m_2 -- ml )/2) 2 tr = ~[
abs(In(fl))
Di = 2x/abs(ln(fl) ) "
(5) (6)
The decision of fl could refer to the heuristic that the smoother the control plane is, the higher the
J.-S. Ker et al. / Fuzzy Sets" and Systems 9l (1997) 53 68
58
[~ is. In this paper, all receptive field functions possess the same variance. Finally, we must solve the composition problem of multidimensional fuzzy receptive field functions. For a multidimensional input problem, the value of each nonzero association cell in FCMAC is denoted as the composite value of receptive field membership grades in respective input dimensions. In this paper, the tensor product operation [8] is adopted for this composition. To explain the composition procedure, the mapping ~(x) to generate the association vector ~ can be further decomposed into three submappings [22]: R:X ~ M,
(7)
Q : M ~ L,
(8)
E:L, M~A.
(9)
The mapping R in Eq. (7) determines the membership grade of any input vector x e X for the receptive field functions. The mapping Q in Eq. (8) is a quantization function that determines the locations of the maximally active (i.e. nonzero) receptive field functions in the input space. The output of Q is a matrix L consisted of maximally active receptive field locations, g
=
[1,1121 ...
I1,1 ... 11,3,
where each column vector l~ (i = 1,2 . . . . . ul contains 2r + 1 elements corresponding to the indices, q i ( x ) - r . . . . . q~(x) . . . . . qi(x) + r, of the receptive field functions with nonzero membership grade. The embedding function E maps the maximally active values by means of evaluating the tensor product of receptive field functions, R (x) = R I (X l ) @ R 2 ( x 2 ) @
..- @ R i ( x i ) @ ...
®R"(x.),
where Ri(xi) = [p)(xi)]j = qi(x) - r . . . . . qi(x) . . . . . qi(x) + r},
onto the association vector ~, that is, c~ = O(X) = E(A,R). In this mapping, a set of active indices of :¢. Indices = ~a l, a2 . . . . . at . . . . . . aN, },
is first selected from elements of the tensor product of maximally active receptive field locations A, i.e., A -11@12@'"@1u.
Then the elements of ~ not addressed by Indices are set to zero, while each of the remaining association cells has a value equivalent to the value of the element, located by the corresponding index in Indices, in R ( X ) . In fact, the active indices in Indices vector also denote the locations of weights which will be retrieved to evaluate the output value. The rule to decide the elements of Indices is called the addressing scheme of FCMAC. Different addressing schemes are distinguished by the strategy of selecting desired element locations from A. Some schemes including the main diagonal, subdiagonal, and anti-diagonal schemes have been discussed in [22]. Besides, a randomly selected addressing scheme was also proposed in [l 1]. In this paper, we employ fully connected addressing scheme for experiments, that is, N L = (2 x r + 1)". If necessary, the other addressing schemes can be chosen for FCMAC model. 2.2. Fuzz), iqference and output generation
After the vector Indices and the association vector ~ are produced, a parallel fuzzy inference process is activated in FCMAC. Each active index a; in Indices is mapped to the address of a corresponding fuzzy weight wj whose membership function is It,,.j. All fuzzy weights are determined at the learning phase, which will be described in next subsection. Then the Mamdani inference scheme [14] is performed for each (~# wj) pair by using :9 as input matching degree and wj as the conclusion part of a fuzzy rule. Therefore, NL partial conclusions, Mi(Y) = ~.i A itwj(y), f o r j = 1 to NL, are produced, and the final fuzzy conclusion can be achieved by executing logical OR operation, that is, N t M ( y ) = V i ~ l M j ( y ) . Finally, the crisp output is produced by performing the COA defuzzification process for the fuzzy conclusion M. 2.3. Learning method
The learning procedure of FCMAC model denotes the procedure of modifying fuzzy weights; the
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53 68 association vector is not adjusted during learning. In FCMAC, the adjustable fuzzy weights are all initialized to zero mean values. The variance of the fuzzy weights are evaluated according to system resolution. During learning, the maximum gradient method is used to decide the adjustment amount of the mean value of each fuzzy weight, mj, and the mean square error (MSE) is adopted as the error function E = ½(O~ - Or) 2,
where Od denotes the desired output and Or denotes the real output. The adjustment procedure is formulated by the following equation:
mj(t + 1)
=
mj(t) + Am~ f o r j = 1 to NL,
where Amj = - rl(c~E/c~mj)and r/is the learning rate whose value is limited between 0 and 1. Therefore, we get Amj = - r/× (Od -- O0 x \~mj
OraL]"
-
q(O d --
Or)
-
~mj/"
According to [6], we know Or = Y ~1 ~im,
ZN, j= 1 '~i
Therefore, we further get =
-
(~ I/Z; ~ 1~imi\ )
-
Step 4. Compute the variance of the input receptive field functions by Eq. (6). Step 5. Do Steps 5.1 and 5.2 until all training patterns have been learned to adjust the fuzzy weights. Step 5.1. Activate FCMAC to get the output Or. Step 5.2. Adjust the mean values of fuzzy weights by Eq. (11). Step 6. Recall all training patterns and calculate the average error, AVE; if AVE is larger than an acceptable error threshold, then go to Step 5; otherwise exit the learning procedure. The variances of fuzzy weights are not adjusted in the proposed learning procedure. According to our experimental results, the strategies of adjustment or not adjustment have almost the same effect.
(10)
Because Od is a constant value, OOd/Ornj always equals zero, so we can eliminate this term from Eq. (10) and obtain Am; =
59
q(Od -- Or)aj ~,NI_ 1 ~ i
F
(11) The whole learning procedure is described as follows:
Step I. The mean values of fuzzy weights are all set to zero. Step 2. Generate training patterns (see next section). Step 3. Compute the mean values of input receptive field functions by Eq. (4).
3. Sampling method for uniformly distributed training patterns According to experimental results, F C M A C will behave very well when the training patterns are uniformly distributed in the whole quantized input space. However, two situations are often encountered during the learning phase. One is the number of training patterns is insufficient. Even if the number of training patterns is enough, the other situation that the training patterns are irregularly distributed may happen. These two situations should be avoided if possible. Unfortunately, it is usually difficult to obtain a set of well-distributed training patterns from environment. Therefore, we develop a sampling method to extract uniformly distributed sample patterns from a set of raw training patterns. There are two phases in the sampling preprocessor: clustering phase and interpolation phase. In the clustering phase, a modified LBG algorithm [13] is adopted to cluster the raw training patterns based on a similarity threshold as the termination condition. In the interpolation phase we use the similarity measure function and fuzzy operations to generate uniformly distributed training patterns.
60
J.-X
K e r et al. / Fuzzy Sets' and Systems 91 (1997) 53 68
3.1. Clustering phase In most problems, the mapping between input patterns and output patterns usually exhibits a common phenomenon that the input patterns in the same cluster will have their corresponding output patterns also in the same cluster. In other words, two training patterns in the same cluster will have similar outputs. Besides, from another viewpoint, we can formulate the mappings between input pattern clusters and output pattern clusters as fuzzy rules to model the mapping relationship between input space and output space. The clustering phase is proposed to find out the different distribution areas of training patterns. In this phase, the number of clusters is an important factor. For the same training data, more clusters cause higher costs, but fewer clusters may result in less precision. So, it needs a trade-off. In this paper, we develop a clustering algorithm derived from the concept of LBG algorithm [13]. There are some advantages of the LBG algorithm. For example, it can cluster the data without giving initial situation. But the original LBG algorithm always generates 2 N clusters. The proposed pattern clustering algorithm is free from this limitation. Moreover, it is unnecessary to decide the number of clusters before the clustering operation is performed. The number of clusters grows incrementally until the goal in data closeness is achieved. The degree of data closeness is evaluated by using a similarity measure function. The shape of similarity measure function dominates the number of clusters to be produced. The terms used in the modified LBG algorithm include (1) 0s: similarity threshold, (2) 0m: movement threshold, (3) 6: a tiny value, (4) c: number of clusters, (5) C / the set of patterns in thejth cluster, (6) # (C j-): number of patterns in the jth cluster, 1 2 i u . (7) Xj, k(=(Xj, k,Xj,k . . . . . X~,R. . . . . Xj, k)). the kth input pattern in the jth cluster, (8) Mr: the mean vector of the jth cluster. In the proposed clustering algorithm, the mean vector, Mj, of the jth cluster of raw training patterns is defined as
1
2
i
u
M i = (mxa, mxj ..... mxj ..... mx r ) •
~#[Cj) vl
where mx~ - /~k= 1 ~'j,k # (cj)
(12)
Besides, Eq. (13) is used to evaluate the average similarity ASj of the jth cluster: #(Cj)
u
/ir.~v
i
~i
I\
S" t ~ S [ ' j-~-r, kl| k-~l i'='1 ~\ n factor/ fl ASi = , # (G) ....
(13)
where 1-[ denotes the absolute function, (-] denotes the minimum operator, Sa(a) = (1 - a)/(1 + 2a), for ,;, ~ ( - 1 , ~ ), denotes the similarity measure function; Fig. 5 illustrates the similarity functions with respect to different 2's, and n_factorl is a normalization factor.
Raw Training Pattern Clustering Algorithm Step 1: Give the values of 0s, 0m, and 6; let c = 1 Step
Step
Step
Step
so that all training patterns are allocated to C1. 2: Compute the mean vector of the first cluster by Eq. (12) and its average similarity value AS1 by Eq. (13). 3: Find a cluster Cj whose average similarity is less than the similarity threshold, i.e., ASj > 0s; If no cluster satisfies the above condition, exit the algorithm. 4: Split C r into two new clusters C'1 and C'2 whose mean vectors are M1 = M r + 6.Mj and Me = Mj - 6.Mj, respectively. 5: Allocate all the training patterns in Cj to either C' 1 or C'2 by the following rule: for each input pattern X~.k, if
i~I S~\ n factor/
> i~i S~
~ac-t~ri "
then allocate Xj.k to C'I, else allocate it to C'2. Step 6: If # ( C ' I ) = 0 or # ( C ' 2 ) = 0 then go back to Step 4 with a different value of fi; otherwise, calculate the mean vectors of C'I and C'2, called M'I and M'2 by applying Eq. (12).
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
Sx(a)
61
1.00
L=-0.9
0.80
0.60
~,=-0.5
0.40
~.=0 0.20
L=10 0.00
I 0.00
I
i
0.20
I 0.40
i
1 0.60
'
I 0.80
'
I 1.00
a Fig. 5. The similarity function Sa(a).
Step 7: If IM'I - M l l + IM'2 - M2I > 0m, let M1 = M'I, M2 = M'2, and go back to Step 5. Step 8: Increase c by one and then go to Step 3 to split the rest clusters.
3.2. Interpolation phase The goal of the interpolation phase is to derive uniformly distributed training patterns from the result of the clustering phase. After the clustering phase is finished, the mean vectors of all clusters have already been decided, and they can be used as the reference patterns in the interpolation phase. Fig. 6 illustrates the interpolation procedure. The basic idea of this phase is to assign a training sample pair to each region of the quantized input space. A training sample is represented by an ordered pair (PI O) of a certain input pattern and its expected output pattern. The input pattern is called the primary part, while the output pattern is called
the resultant part. For example, the training pairs in the color calibration problem have the format (r, 9, b lAr, Ag, Ab), where P = (r, 9, b) is the primary part and O = (Ar, A9, Ab) is the resultant part. Since we want to generate uniformly distributed training patterns over the whole quantized input space, the primary parts of training sample pairs can be trivially decided by adopting each quantization region in the input space as the primary part of a training pair. As illustrated in Fig. 6, to decide the corresponding resultant part of a training pair, the first step is to compute the similarity between the primary part of training sample pair and each cluster. The second step is to check if the similarity value can pass a threshold test for each cluster. The third step is to evaluate the desired resultant part of the training sample pair. Basically, the resultant part is generated by the weighted mean approach for the resultant parts of the mean vectors of all clusters passing the threshold test. Assume the mean vector of the jth cluster is denoted as CMj, where CM~ is expressed as the
62
J.-X Ker et al. , Fuzzy Sets" and Systems 91 (1997) 53 68
primary part of the clustered mean vectors
m
resultant part of the clustered mean vectors
:
:
2
~
Iu,,ctioo [I l [operotorjL,uoctlon,
,-t(-Mean
~
Weighted
~
(5)
primary part of sample pattern
(6)
interpolated training sample pair (pJ,...,p',...,p"[ ot,...,o',...,o')
(P",...,P',.-.,V')
Fig. 6. The sampling flow of interpolation phase.
Step 4: Calculate the resultant part, O, of the train-
following form:
ing sample pair; the weighted mean approach is applied, as expressed in Eq. (17):
CM.i = (piloi) = (mp) .... , mpii ..... m p i l m o l ..... moii ..... moil, where u and v denote the number of input/output dimensions, respectively. The whole interpolation procedure to generate the training sample pair (PtO) for each quantized region is explained as follows: Step
1: Evaluate the corresponding similarity measure, s~, between the primary part of training sample pair, P, and the mean vector of each cluster C j on each input dimension i by Eq. (14):
sj =
t~act-0rl
(14)
Step 2: Decide the total similarity between P and the mean vector of each cluster Cj by Eq. (15), tsj =
15)
Sj. i-1
o' = 5~.~=t TH(tsj) × ,s~× mo~ 25-1TH(tsfl x s~ f o r i = 1 to v.
(17)
Step 5: Repeat Steps 1-4 until all quantized regions are explored.
4. Experimental results Two experiments, function approximation of sinusoid functions and color calibration, will be discussed in this section. In the former experiment, the effects of standard CMAC, BMAC (higherorder B-spline CMAC), and FCMAC on various conditions are compared. In both experiments, the simulation process is divided into learning phase and recall phase. During the learning phase, the total absolute e r r o r , ~trainingpatt . . . . HO~ - Or tl, is defined as the error function. The training pattern sampling procedure in last section is invoked as the preprocessing task for the learning phase.
Step 3: Perform the threshold test for the similarity 4.1. Function approximation for sinusoid fimctions
degree of each cluster C; by Eq. (16t, TH(tsi) = •
10 if tsi/> OK, otherwise.
(16)
In this experiment, two sinusoid functions,
y = sin(nx/180) and z = sin(nx/180)sin(ny/180), 0 <~x,
J.-S. Ker et al. / F u z ~ Sets and Systems 91 (1997) 53-68 1.00--
1,00
050-
0.50 .
/
000-
-050.
-1 O0
I
'
I
1 O0
200
'
I
63
--
/
0.00 -
-050 .
X !
'
300
-1.00
400
,
0
,
1O0
(a)
,
2 O0
|
3 O0
X ,
! 400
(b) 1 O0
oq/ \ 000.
0.50
/
.
0.00
-050
X I
I
I
1 O0
200
300
(c)
! 400
x
-1 O0 0
100
200
300
400
(d)
Fig. 7. The approximated curves of sinusoid function by CMAC, BMAC, and FCMAC.
y ~< 360, are approximated by CMAC, BMAC, and F C M A C models. In this experiment, each input dimension is quantized into 20 intervals, and each quantization region is assigned with a training sample pair. Fig. 7 shows the output curves ofy = sin(rtx/180) approximated by CMAC, BMAC, and F C M A C models. The configuration for CMAC is c = 4; for BMAC is n = G = 4; and for F C M A C is r = 2 with fully connected addressing scheme. Fully connected addressing scheme is employed in both BMAC and FCMAC model, so, NL = 4" and 5", respectively. Obviously, the output curve of CMAC in Fig. 7(a) is staircased, which is far away from the desired path. The output curve approximated by BMAC, as shown in Fig. 7(b), is almost the same as the
desired curve. Figs. 7(c) and (d) show the output curves approximated by FCMAC with fl = 0.5 and fl = 0.8, respectively. We can see that the greater value of fl makes the smoother output result. Obviously, FCMAC and BMAC models are much better than standard CMAC model in function approximation. Fig. 8 illustrates the absolute error curves for y = sin(nx/180) approximated by CMAC, FCMAC, and BMAC with learning rate r / = 1.0 and 0.8. A training iteration means that all training samples have updated the fuzzy weights one time. It is obvious that FCMAC has good convergence rate than the other two. Finally, the effects on the convergence speed with respect to different neighborhood parameters (c of
64
J.-S. K e r et al. i/ Fuzzy Sets and Systems 91 (1997) 5 3 - 6 8
16.00 -I total [ absolute 1 ~ error 12.00
s oo
11=10
m
19
I ~ 4,00 I
B
~
~
............
~
16.00 ~ total / absolute error 12.00
tMA~:
I 4.00
~. ~
0
_-_7-_- ~.S~
800
I,LMAt
I,..\ \ \
0.00
no. of iterations 4
rl=0.8
8
12
16
'f{ ~,~
I
r
0
4
000
20
FCMAC
"
t
I
8
12
-
(a)
110 o f
I 16
iterations I 20
(b)
Fig. 8. The absolute error curves for the sinusoid function approximated by CMAC, BMAC, and FCMAC.
100 -
no. of iterations /\
80
\
/
\
/
60
CMAC /
. . . . . . .
/
40_
• -
BMAC FCMAC
I!
I
tt
I I
20
" "
t
!
/
l~=0.8
,, . . . . . . .
,, "
CMAC: c BMAC: (; FCMAC : r
""
f~=0.7
/
0
'
0
I
'
4
I
8
'
I
size of neighborhood
12
Fig. 9. The stability curves for CMAC, BMAC, and FCMAC.
C M A C , G of B M A C , and r of F C M A C , respectively) are evaluated. The termination condition for each model is in the iteration that the absolute error of each training sample is smaller than a given threshold. Fig. 9 illustrates the statistic results with 0.01 threshold. Obviously, F C M A C model exhibits better stability than the other two. The simulation results for approximating the twoinput sinusoid function, z = sin(rtx/180) sin(~y/180), by C M A C , B M A C , and F C M A C models indicate
that the same conclusions can be reached. F C M A C exhibits better stability and convergence speed than the other two models. Fig. 10 illustrates the individual output responses for these three models. 4.2. Color calibration
As shown in Fig. 11, the operation flow of a color reproduction system can be briefly described as follows. First, the original color pictures are digitized
65
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
(a)
(b)
.:z
,,.
(c) "
K'X
•
~.)C
Desire output : z - sm(1-~6). sm(1-~) CMAC : q=20, c=4 BMAC : q=20, n=4, G=4, NL=I6 FCMAC : q=20, r=2, NL=25 number of training patterns: 400 Fig. 10. The approximated output of a two-input sinusoid function by CMAC, BMAC, and FCMAC.
Original ~ color image (o)
FCMAC [i
I Printed color calibration]~ printer ]-~ image (P)
co,o, i
Fig. 11. The role of color calibration in a color reproduction system.
by a scanning device and stored as image files in either hard-disk or CD-ROM. Then the digitized image files can be processed with desired image processing operations. Finally, to obtain real duplications of the processed image files, these image files have to be printed out by color printing devices. Due to the nonlinear mapping relationship between the scanning and printing devices, we need a color calibration mechanism to make the colors in reproduced image as identical as those of original one. In this experiment, F C M A C is adopted as the color calibration mechanism. In our experiment, the scanning device is the UMAX UC300
color scanner, and the printing device is the F A R G O PrimeraPro thermal color printer. There are two major parameters, 0s and 2, to be determined before F C M A C can properly calibrate color images. We can see that the image with 0s = 1.0 is almost the same as the uncalibrated image. The reason is described as follows. To have a nonzero resultant part (i.e., nonzero color calibration amount), the training sample pairs generated by the proposed sampling procedure with 0s = 1.0 must be very similar to one of the raw training patterns. Obviously, in this case, there are very few sample patterns satisfying this requirement. As
J.-S. Ker et al. / Fuzzy Sets and S.vstems 91 (1997) 53-68
66 80.00
]RMSE I ;~=-0.9
60.00
X=-0.5
L=O.O
X=2.0
40.00
~L=I0.0 20.00
gs 0.00 0.00
0.20
0.40
0.60
0.80
1.00
Fig. 12. The R M S E curves with respect to different 0~ and ,L
a consequence, very few pixels in the reproduced image can be calibrated, so that the calibrated image looks like the uncalibrated one. On the contrary, in the case of 0s = 0, almost all the clusters can pass the threshold test to take part in the decision of the resultant part of a training sample pair. In other words, each training sample pair will be affected by most raw training patterns in generating the resultant part. Therefore, the calibrated image may appear improper calibration effect due to overlearning. Fig. 12 illustrates the curves of root mean square error (RMSE) with respect to different settings of 05 and 2. Finally, we adopt 05 = 0.9 and ). = 2.0 as the parameters to extract training samples. The FCMAC model is configured with three inputs, and three outputs; the quantization resolution is set to 10 on each input dimension, and the learning rate q is 0.5. Fig. 13 shows the RMSE curve for training with color calibration training sample pairs. It indicates that FCMAC has a fast convergence speed in adjusting the fuzzy weights to approximate the nonlinear relationship between scanning and printing devices. Fig. 14 shows two experimental images
to illustrate the capability of FCMAC in color calibration. Figs. 14(a) and (b) present the reproduced images without applying any color calibration scheme. They do not look like the original pictures. Figs. 14(c) and (d) illustrate the reproduced images when FCMAC is invoked. Obviously, the colors presented in these calibrated images are more vivid and realistic.
5. Conclusions
FCMAC can be applied to many application fields to approximate the nonlinear control function of target systems. It is based on a local weight updating scheme, which has several advantages than the global weight updating scheme used by backpropagation neural networks, such as the features of fast learning and insensitivity to the order of training patterns. The information stored in the weight memory of FCMAC are fuzzy numbers instead of crisp values, so it stores fuzzy associations with distributed encoding, and manipulates the stored information by using fuzzy set theory. Based
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
67
RMSE
6o.oo
60.00
40.00
20.00
no. of iterations o.oo
I
0
4
'
8
I '-
12
'
I
'
16
I
20
Fig. 13. The convergence curve of F C M A C during color calibration learning.
(a) Image 1 - reproduced w i t h o u t calibration
(b) Image 2 - reproduced w i t h o u t calibration
(c) Image 1 - calibrated by FCMAC
(d) Image 2 - calibrated by FCMAC
Fig. 14. Calibration results by FCMAC. For a color version of this figure see URL: http://www.elsevier.com/locate/fss
68
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53 68
on the B M A C architecture, F C M A C blindly extracts a n d refines fuzzy rules from t r a i n i n g samples. F r o m the s i m u l a t i o n results of a p p r o x i m a t i n g sinusoid functions, we can k n o w that F C M A C has a good convergence rate a n d the o u t p u t plane of the controlled plant is s m o o t h e r than C M A C model. Besides, the optimal values of the neighborhood p a r a m e t e r c of C M A C a n d G of B M A C are hard to be selected, but the F C M A C has a good stability on the radius p a r a m e t e r r. This can reduce the design effort of F C M A C . F u r t h e r m o r e , the bell-shaped receptive field functions in F C M A C is easier to be m a n i p u l a t e d than the recurrent Bspline receptive field functions in BMAC. F o r the color calibration experiment, the parameters of 0s a n d ,:. in the s a m p l i n g procedure will affect the calibration results. A higher value of 0s will cause the lower calibration efficiency. The p a r a m e t e r 2 presents similar effect on color calibration as 0,. F r o m the visual perception of the calibrated images, we can conclude that F C M A C can result in excellent color r e p r o d u c t i o n on color images.
References [1] J.S. Albus, A theory of cerebellar function, Math. Biosci. 10 (1971) 25-61. [2] J.S. Albus, A new approach to manipulator control: the cerebellar model articulation controller, J. Dyn. System Measurement Control 97 (1975) 220-227. [3] J.S. Albus, Data storage in the CMAC, J. Dyn. System Measurement Control 97 (1975) 228 233. [4] J.S. Albus, Mechanisms of planning and problem solving in the brain, Math. Biosci. 45 (1979) 247 293. [5] J.S. A l b u s , B r a i n s , Behavior and Robotics (BYTE/McGraw-Hilt, Peterborough, NH, 1981) 143 186. [6] M. Braae and D.A. Rutherford, Fuzzy relations in a control setting, Kybernetes 7 (1978) 185-188. [7] C.W. Chang and P.R. Chang, Neural plant inversecontrol approach to color error reduction for scanner and printer, Proc. Internat. Conf. on Neural Networks (1993) 1979-1983. [8] P.J. Hartley, Tensor product approximations to data defined on rectangular meshes in N-space, Comput. J. 19 (1974) 348-351.
[9] D. Hubel and T.N. Wiesel, Receptive fields, binocular interaction and functional architecture in cat's visual cortex, J. Physiol. 160 (1962). [10] H.R. Kang and PG. Anderson, Neural network application to the color scanner and printer calibrations, J. Electron. Imaging I (1992) 125-135. [11] J.S. Ker, Y.H. Kuo and B.D. Liu, Hardware realization of higher-order CMAC model for color calibration, Proc. IEEE lnternat. Conj'. on Neural Networks (Perth, Australia, 1995) 1656-1661. [12] A. Kolcz and N.M. Allinson, Application of the CMAC input encoding scheme in the N-tuple approximation network, lEE Proc. Comput. Digital Techniques 141 0994) 177 183. [13] Y. Linde, A. Buzo and R.M. Gray, An algorithm for vector quantizer design, IEEE Trans. Commun. COM-28 (1980) 84 95. [14] E.H. Mamdani and S. Assilian,An experiment in linguistic synthesis with a fuzzy logic controller, lnternat. J. ManMachine Stud. 7 (1975) 1-13. El5] S. Manglevhedakar, An adaptive hierarchical model for computer vision, Thesis, Louisiana State Univ. (1986). [16] W.T. Miller, R.P. Hewes, F.H. Glanz and L.G. Kraft, Real-time dynamic control of an industrial manipulator using a neural-network-based learning controller, IEEE Trans. Robot. Automat. 6 (1990) 1--9. [17] J. Moody and C, Darken, Learning with localized receptive fields, Proc. Connectionist Model Summer School (1988). [18] T. Poggio and F. Girosi, Networks for approximation and learning, Proc. IEEE 78 (1990) 1481-1496. [19] J.C. Smith, D.C, Tribolet, H.E. Mostafa and E. Maghakian, Development of color graphics printer, IEEE Trans. Consumer Electron. 34 (1988) 523 529. [20] D.L. Sparks and J.S. Nelson, Sensory and motor maps in the mammalian superior colliculus, Trends Neurosci. 10 (1987) 312 317. [21] G.K. Starkweather, A color-correction scheme for color electric printers, Color Res. Appl. (Suppl.) 11 (1994) 108 114. [22] H.L. Stephen, A.H. David and J.G. Jack, Theory and development of higher-order CMAC neural networks, IEEE Control Systems 12 (2) (1992) 23-30. [23] S. Suzuki, T. Kusunoki and M. Mori, Color characteristic design for color scanners, Appl. Optics 29 (34) (1990) 5187-5192. [24] P.Z. Wang and H.M. Zhang, Truth-valued flow inference and its dynamic analysis, Proc. Peking Normal University, 1 (1989).
FUZilY
sets and systems Fuzzy Sets and Systems91 (1997) 53-68
ELSEVIER
A fuzzy CMAC model for color reproduction Jar-Shone
K e r a, C h a o - C h i h
H s u b, Y a u - H w a n g
K u o b'*, B i n - D a L i u a
"Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan, ROC blnstitute of Information Engineering, National Cheng Kung University, Tainan, Taiwan, ROC
Received September 1995;revised February 1996
Abstract
Color reproduction is a complex nonlinear mapping problem due to gamut mismatch, resolution conversion, quantization, nonlinear color relationship between scanner and printer. To solve such a complex problem in color reproduction, this paper proposes a fuzzy CMAC model, which adopts a special parallel fuzzy inference-like process to realize the function similar to higher-order CMAC. In this model, recursive B-spline receptive field functions are replaced by fuzzy sets with bell-shaped membership function, and the weights to evaluate output values are also not crisp values but fuzzy sets. The learning algorithm is based on the maximum gradient method. For the situations of insufficiently or irregularly distributed training patterns, this paper develops a sampling method to generate uniformly distributed training patterns. According to experimental results, the proposed fuzzy CMAC model has shown its effectiveness on color reproduction and general function approximations. Besides, it has advantages of fast learning speed, simple computation, and high stability on model parameters. © 1997 Elsevier Science B.V. Keywords: Fuzzy logic; Cluster analysis; CMAC; Color calibration
1. Introduction
Due to the rapid progress on the technologies of computer and color scanning/printing devices, the extraction, manipulation, and reproduction of color documents have become interesting research topics. One of the major problems among these topics is how to reproduce the original color information of documents or images when they are scanned in and then printed out. Unfortunately, the
*Fax : 886 6 274 7076; e-mail: [email protected].
color mismatch phenomenon is very severe and undesired in the existing color reproduction systems [10, 21]. The color mismatch may result from the nonlinear characteristics, for example, gamut mismatch, resolution conversion and quantization, between the scanning devices and the printing devices [19, 23]. The principal purpose of color reproduction is to maintain color resemblance between the reproduced and original images, and thus it is a problem of nonlinear function approximation. Conventionally, most color calibration systems are based on a table look-up scheme, where the content of the look-up table is manually defined. Such an approach requires huge design effort and lacks self-adaptivity.
0165-0114/97/$17.00 © 1997 ElsevierScienceB.V. All rights reserved PII S0165-01 14(96)00083-8
54
,Z-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53 68
To develop a color calibration model with selflearning and self-adaptivity capabilities, Chang [7] and Kang [10] proposed, respectively, an approach that uses multilayer feedforward network with backpropagation learning algorithm to calibrate color mismatch. However, their methods may have some drawbacks such as low learning speed and high sensitivity to the order of presentation of the training data. Unlike the global weight updating scheme of backpropagation neural networks, Cerebellar Model Articulation Controller (CMAC) model, proposed by Albus [1-5], is characterized by a local weight updating scheme which exhibits the advantages of fast learning and high convergence rate. The higher-order CMAC with B-spline receptive field functions [22] can further improve the accuracy of function approximation. CMAC has been successfully applied in many fields such as robot control [-16], pattern recognition [15], and signal processing [12]. Recently, it is also adopted for color calibration [11]. The operation of CMAC model can be considered as a mechanism whose output is obtained by adding a fixed number of weights. The input vector is transformed into an Na-dimensional association vector in which each element is associated with an individual weight cell. There are always N~. nonzero elements in the association vector so that only NL crisp weights contribute to the evaluation of an output value. There is one significant criterion in determining which weight cells are to be activated for a given input vector. The number of overlapped weights for two input vectors is inversely proportional to their distance (e.g., the hamming distance). If the distance exceeds a predefined value, there will be no overlapped weight between the input-correlated association vectors. Such a feature is the well-known local generalization characteristic of CMAC-based models. But there exists some problems when one CMAC-based model is applied. First, CMAC will be computationally complicated when the order of B-spline receptive field functions is high. Second, the convergence speed of CMAC is more or less unstable as the model parameters alter, which will increase the design effort in choosing a set of appropriate parameters. Therefore. this paper intends to
develop a new model for color calibration, which can overcome the mentioned problems. Embedding fuzzy inference capability into conventional CMAC model may be a good alternative to reach the goal of this paper. Thus, a novel fuzzy CMAC model, called FCMAC, is proposed in this paper to fully exploit the advantages of fuzzy set theory and the local generalization feature of CMAC model. To reserve the characteristics of CMAC, FCMAC performs a special parallel fuzzy inference process which has the following features: (1) the number of fuzzy rules fired at each cycle to get an output value is fixed; (2) the decision of the fuzzy rules to be fired and their firing strength is realized by the mapping from input vector to association vector; (3) the evaluation of output values corresponds to three operations: parallel rule firing, conflict resolution, and defuzzification. Besides, this paper also proposes a training pattern sampling algorithm, based on fuzzy clustering, similarity measure, and interpolation approaches, to generate a set of uniformly distributed training patterns from raw training data. This sampling algorithm is helpful in improving the accuracy of function approximation with FCMAC, especially when the raw training data are insufficient or irregularly distributed. This paper is organized as follows. Section 2 presents the FCMAC model. Section 3 explains the sampling algorithm for generating a set of uniformly distributed training patterns. Section 4 discusses the experimental results. Finally, some conclusions are given in Section 5.
2. Fuzzy CMAC model Fig. 1 shows the structure of FCMAC. Similar to conventional CMAC model, FCMAC approximates a nonlinear function y = f ( x ) by using two primary mappings: 4s:X ~ A,
(1)
O: A ~ D,
(2)
where X is a continuous s-dimensional input space, A is an NA-dimensional association space, and D is
55
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
fuzzy weights
I
,!
Desired actuatorsignal ~x
Adju~ activeweights(~
/
~
~
Nonzero Excitation
.... X : Input
A : Association
Actuator
•
Zero Excitation
D : Defuzzification
Fig. 1. Structure of FCMAC model.
a one-dimensional output space. But these two mappings are realized by fuzzy operations unlike conventional CMAC models. The function ~k(x) is usually fixed and maps each point, x, in the input space X onto an association vector ~ e A that always has NL nonzero elements. Formally, we can write it as = ~(x)
= (~t,~2 ....
,~j,
. . . , ~N,),
where ~i is derived from the composition of fuzzy receptive field functions and sensory inputs, and 0 ~< ~j <~ 1 for all components in ~. The second mapping, 0(~), computes a scalar output. In fact, it is a special mapping which executes a parallel fuzzy inference-like procedure. In this procedure, each nonzero association cell ~j retrieves a corresponding weight cell wj which is a fuzzy number. Moreover, each (~j, wj) pair constitutes the inference channel of a fuzzy rule, where ~j is the matching degree of the condition part while wj is the conclusion part of that rule. In other words, there are always NL(~j, wi) fuzzy rules activated in each FCMACs recall cycle to get final output. The mapping of 0(~), that is, the parallel fuzzy inference procedure, can be interpreted by employ-
ing the truth-value flow model 1-24], as shown in Fig. 2. First, the sensory input vector x is used to decide the addresses of nonzero association cells and calculate the values of those nonzero association cells. The addresses are used to retrieve Nt corresponding fuzzy weights. Then each fuzzy weight is inferred to produce a partial fuzzy output by applying the value of its corresponding association cell as input matching degree (rule firing strength). Third, all NL partial fuzzy outputs are aggregated to a global fuzzy conclusion by a certain conflict resolution strategy. Finally, the global fuzzy conclusion is defuzzified into a scalar output by the centroid of area (COA) approach. FCMAC model is basically a special table lookup model in which fuzzy weights are stored and manipulated by means of fuzzy inference. It does not operate by mathematically analyzing the dynamics of the controlled plant and then solving equations, but computes the control amount by referring to a fuzzy weight memory table. The weight memory table is established by storing the mappings between input values and control amounts. FCMAC learns the control function over the input space by modifying or generalizing its internal data distribution obtained from previous
56
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
Matching degree calculation
Partial fuzzy outputs Conflict resolution
M = vM
---'q
Defuzzification
i
. IM,(y) o, I
('F Fig. 2. The truth-value flow model of fuzzy inference.
learning. Once the fuzzy weight memory table is built, the control amount is then obtained by executing the two mappings represented by Eqs. (1) and (2).
2.1. Fuzzy receptive field functions It is well-known that many biological sensorimotor control structures in the brain are organized in a way that neurons possess locally tuned overlapping receptive fields [9,20]. Several computational paradigms of neural network, such as CMAC and radial basis functions (RBF) [17, 18], have been quite successful in representing complex nonlinear functions by using similar organization principles. These networks construct local approximation to multi-input/multi-output functions, which is similar to the approximation scheme of using piecewise polynomials or other basis functions on a set of training data. In FCMAC model, we adopt normal bell-shaped membership functions as the receptive field functions:
Before further discussing, some terms used in this paper must be declared: i ( = 1..... u), the index of input dimension being considered; x (=(Xl, x2 ..... xi ..... xu)), input vector; X .... ~, the upper bound of dimension i; Xmin.i, the lower bound of dimension i; Rg, the resolution of dimension i; Di, the width of quantization interval on dimension i; r, the effective radius of the receptive field function by using Di as unit; and q~(x), the quantized position of input vector x on dimension i. The quantized position of input vector x on dimension i can be determined by the following equation:
qi(x) = [ x l
Di
where L" J denotes the mathematical floor function. Therefore, when the bounded input space and the width of quantization interval are given, the quantized resolution on dimension i can be computed as
/~(x) = e -((x-m)/~)~, where m represents the mean value and ~ represents the variance.
_1
....
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
57
u i 1
....
I! X
m in,i':
S
m iI
m
i J
m
m ux,i
i q
Fig. 3. Fuzzy receptivefield functions. 1.00
_
P
0.00 i
mj
i mj+l
Fig. 4. Decision of the receptive field variance. Now, we derive the bell-shaped receptive field function in Eq. (3): #}(x) = e -((x'-m))/')2,
(3)
where m} denotes the jth mean on dimension i, which can be expressed by the following formula: mji = X m i n , i + (j _ ½) x Di
where j E I-- r + 1, Ri + r].
(4)
The receptive field function #~(x) denotes the grade of x belonging to interval [-2j_ i 1,2j)i where /~j-- 1 = (qi(x) - 1) × Di and 2~ = qi(x) x Di. When xi is in the middle point of that interval, it will have the full membership of p~(x). Fig. 3 illustrates an example of the receptive field functions of F C M A C model.
The decision of the variance tr of #~(x) is based on the idea that allocates any two adjacent fuzzy receptive field functions to have a proper overlapping region. Assume that by this arrangement the two adjacent receptive field functions will intersect at the point whose membership grade is fl ( ~ (0, 1]), as illustrated in Fig. 4. Then a, expressed in Eq. (6), can be easily derived from Eq. (5): fl = e x p ( _
( ((ml + m 2 - ) a / 2 ) - - m l ) 2 ) ,
/((m_2 -- ml )/2) 2 tr = ~[
abs(In(fl))
Di = 2x/abs(ln(fl) ) "
(5) (6)
The decision of fl could refer to the heuristic that the smoother the control plane is, the higher the
J.-S. Ker et al. / Fuzzy Sets" and Systems 9l (1997) 53 68
58
[~ is. In this paper, all receptive field functions possess the same variance. Finally, we must solve the composition problem of multidimensional fuzzy receptive field functions. For a multidimensional input problem, the value of each nonzero association cell in FCMAC is denoted as the composite value of receptive field membership grades in respective input dimensions. In this paper, the tensor product operation [8] is adopted for this composition. To explain the composition procedure, the mapping ~(x) to generate the association vector ~ can be further decomposed into three submappings [22]: R:X ~ M,
(7)
Q : M ~ L,
(8)
E:L, M~A.
(9)
The mapping R in Eq. (7) determines the membership grade of any input vector x e X for the receptive field functions. The mapping Q in Eq. (8) is a quantization function that determines the locations of the maximally active (i.e. nonzero) receptive field functions in the input space. The output of Q is a matrix L consisted of maximally active receptive field locations, g
=
[1,1121 ...
I1,1 ... 11,3,
where each column vector l~ (i = 1,2 . . . . . ul contains 2r + 1 elements corresponding to the indices, q i ( x ) - r . . . . . q~(x) . . . . . qi(x) + r, of the receptive field functions with nonzero membership grade. The embedding function E maps the maximally active values by means of evaluating the tensor product of receptive field functions, R (x) = R I (X l ) @ R 2 ( x 2 ) @
..- @ R i ( x i ) @ ...
®R"(x.),
where Ri(xi) = [p)(xi)]j = qi(x) - r . . . . . qi(x) . . . . . qi(x) + r},
onto the association vector ~, that is, c~ = O(X) = E(A,R). In this mapping, a set of active indices of :¢. Indices = ~a l, a2 . . . . . at . . . . . . aN, },
is first selected from elements of the tensor product of maximally active receptive field locations A, i.e., A -11@12@'"@1u.
Then the elements of ~ not addressed by Indices are set to zero, while each of the remaining association cells has a value equivalent to the value of the element, located by the corresponding index in Indices, in R ( X ) . In fact, the active indices in Indices vector also denote the locations of weights which will be retrieved to evaluate the output value. The rule to decide the elements of Indices is called the addressing scheme of FCMAC. Different addressing schemes are distinguished by the strategy of selecting desired element locations from A. Some schemes including the main diagonal, subdiagonal, and anti-diagonal schemes have been discussed in [22]. Besides, a randomly selected addressing scheme was also proposed in [l 1]. In this paper, we employ fully connected addressing scheme for experiments, that is, N L = (2 x r + 1)". If necessary, the other addressing schemes can be chosen for FCMAC model. 2.2. Fuzz), iqference and output generation
After the vector Indices and the association vector ~ are produced, a parallel fuzzy inference process is activated in FCMAC. Each active index a; in Indices is mapped to the address of a corresponding fuzzy weight wj whose membership function is It,,.j. All fuzzy weights are determined at the learning phase, which will be described in next subsection. Then the Mamdani inference scheme [14] is performed for each (~# wj) pair by using :9 as input matching degree and wj as the conclusion part of a fuzzy rule. Therefore, NL partial conclusions, Mi(Y) = ~.i A itwj(y), f o r j = 1 to NL, are produced, and the final fuzzy conclusion can be achieved by executing logical OR operation, that is, N t M ( y ) = V i ~ l M j ( y ) . Finally, the crisp output is produced by performing the COA defuzzification process for the fuzzy conclusion M. 2.3. Learning method
The learning procedure of FCMAC model denotes the procedure of modifying fuzzy weights; the
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53 68 association vector is not adjusted during learning. In FCMAC, the adjustable fuzzy weights are all initialized to zero mean values. The variance of the fuzzy weights are evaluated according to system resolution. During learning, the maximum gradient method is used to decide the adjustment amount of the mean value of each fuzzy weight, mj, and the mean square error (MSE) is adopted as the error function E = ½(O~ - Or) 2,
where Od denotes the desired output and Or denotes the real output. The adjustment procedure is formulated by the following equation:
mj(t + 1)
=
mj(t) + Am~ f o r j = 1 to NL,
where Amj = - rl(c~E/c~mj)and r/is the learning rate whose value is limited between 0 and 1. Therefore, we get Amj = - r/× (Od -- O0 x \~mj
OraL]"
-
q(O d --
Or)
-
~mj/"
According to [6], we know Or = Y ~1 ~im,
ZN, j= 1 '~i
Therefore, we further get =
-
(~ I/Z; ~ 1~imi\ )
-
Step 4. Compute the variance of the input receptive field functions by Eq. (6). Step 5. Do Steps 5.1 and 5.2 until all training patterns have been learned to adjust the fuzzy weights. Step 5.1. Activate FCMAC to get the output Or. Step 5.2. Adjust the mean values of fuzzy weights by Eq. (11). Step 6. Recall all training patterns and calculate the average error, AVE; if AVE is larger than an acceptable error threshold, then go to Step 5; otherwise exit the learning procedure. The variances of fuzzy weights are not adjusted in the proposed learning procedure. According to our experimental results, the strategies of adjustment or not adjustment have almost the same effect.
(10)
Because Od is a constant value, OOd/Ornj always equals zero, so we can eliminate this term from Eq. (10) and obtain Am; =
59
q(Od -- Or)aj ~,NI_ 1 ~ i
F
(11) The whole learning procedure is described as follows:
Step I. The mean values of fuzzy weights are all set to zero. Step 2. Generate training patterns (see next section). Step 3. Compute the mean values of input receptive field functions by Eq. (4).
3. Sampling method for uniformly distributed training patterns According to experimental results, F C M A C will behave very well when the training patterns are uniformly distributed in the whole quantized input space. However, two situations are often encountered during the learning phase. One is the number of training patterns is insufficient. Even if the number of training patterns is enough, the other situation that the training patterns are irregularly distributed may happen. These two situations should be avoided if possible. Unfortunately, it is usually difficult to obtain a set of well-distributed training patterns from environment. Therefore, we develop a sampling method to extract uniformly distributed sample patterns from a set of raw training patterns. There are two phases in the sampling preprocessor: clustering phase and interpolation phase. In the clustering phase, a modified LBG algorithm [13] is adopted to cluster the raw training patterns based on a similarity threshold as the termination condition. In the interpolation phase we use the similarity measure function and fuzzy operations to generate uniformly distributed training patterns.
60
J.-X
K e r et al. / Fuzzy Sets' and Systems 91 (1997) 53 68
3.1. Clustering phase In most problems, the mapping between input patterns and output patterns usually exhibits a common phenomenon that the input patterns in the same cluster will have their corresponding output patterns also in the same cluster. In other words, two training patterns in the same cluster will have similar outputs. Besides, from another viewpoint, we can formulate the mappings between input pattern clusters and output pattern clusters as fuzzy rules to model the mapping relationship between input space and output space. The clustering phase is proposed to find out the different distribution areas of training patterns. In this phase, the number of clusters is an important factor. For the same training data, more clusters cause higher costs, but fewer clusters may result in less precision. So, it needs a trade-off. In this paper, we develop a clustering algorithm derived from the concept of LBG algorithm [13]. There are some advantages of the LBG algorithm. For example, it can cluster the data without giving initial situation. But the original LBG algorithm always generates 2 N clusters. The proposed pattern clustering algorithm is free from this limitation. Moreover, it is unnecessary to decide the number of clusters before the clustering operation is performed. The number of clusters grows incrementally until the goal in data closeness is achieved. The degree of data closeness is evaluated by using a similarity measure function. The shape of similarity measure function dominates the number of clusters to be produced. The terms used in the modified LBG algorithm include (1) 0s: similarity threshold, (2) 0m: movement threshold, (3) 6: a tiny value, (4) c: number of clusters, (5) C / the set of patterns in thejth cluster, (6) # (C j-): number of patterns in the jth cluster, 1 2 i u . (7) Xj, k(=(Xj, k,Xj,k . . . . . X~,R. . . . . Xj, k)). the kth input pattern in the jth cluster, (8) Mr: the mean vector of the jth cluster. In the proposed clustering algorithm, the mean vector, Mj, of the jth cluster of raw training patterns is defined as
1
2
i
u
M i = (mxa, mxj ..... mxj ..... mx r ) •
~#[Cj) vl
where mx~ - /~k= 1 ~'j,k # (cj)
(12)
Besides, Eq. (13) is used to evaluate the average similarity ASj of the jth cluster: #(Cj)
u
/ir.~v
i
~i
I\
S" t ~ S [ ' j-~-r, kl| k-~l i'='1 ~\ n factor/ fl ASi = , # (G) ....
(13)
where 1-[ denotes the absolute function, (-] denotes the minimum operator, Sa(a) = (1 - a)/(1 + 2a), for ,;, ~ ( - 1 , ~ ), denotes the similarity measure function; Fig. 5 illustrates the similarity functions with respect to different 2's, and n_factorl is a normalization factor.
Raw Training Pattern Clustering Algorithm Step 1: Give the values of 0s, 0m, and 6; let c = 1 Step
Step
Step
Step
so that all training patterns are allocated to C1. 2: Compute the mean vector of the first cluster by Eq. (12) and its average similarity value AS1 by Eq. (13). 3: Find a cluster Cj whose average similarity is less than the similarity threshold, i.e., ASj > 0s; If no cluster satisfies the above condition, exit the algorithm. 4: Split C r into two new clusters C'1 and C'2 whose mean vectors are M1 = M r + 6.Mj and Me = Mj - 6.Mj, respectively. 5: Allocate all the training patterns in Cj to either C' 1 or C'2 by the following rule: for each input pattern X~.k, if
i~I S~\ n factor/
> i~i S~
~ac-t~ri "
then allocate Xj.k to C'I, else allocate it to C'2. Step 6: If # ( C ' I ) = 0 or # ( C ' 2 ) = 0 then go back to Step 4 with a different value of fi; otherwise, calculate the mean vectors of C'I and C'2, called M'I and M'2 by applying Eq. (12).
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
Sx(a)
61
1.00
L=-0.9
0.80
0.60
~,=-0.5
0.40
~.=0 0.20
L=10 0.00
I 0.00
I
i
0.20
I 0.40
i
1 0.60
'
I 0.80
'
I 1.00
a Fig. 5. The similarity function Sa(a).
Step 7: If IM'I - M l l + IM'2 - M2I > 0m, let M1 = M'I, M2 = M'2, and go back to Step 5. Step 8: Increase c by one and then go to Step 3 to split the rest clusters.
3.2. Interpolation phase The goal of the interpolation phase is to derive uniformly distributed training patterns from the result of the clustering phase. After the clustering phase is finished, the mean vectors of all clusters have already been decided, and they can be used as the reference patterns in the interpolation phase. Fig. 6 illustrates the interpolation procedure. The basic idea of this phase is to assign a training sample pair to each region of the quantized input space. A training sample is represented by an ordered pair (PI O) of a certain input pattern and its expected output pattern. The input pattern is called the primary part, while the output pattern is called
the resultant part. For example, the training pairs in the color calibration problem have the format (r, 9, b lAr, Ag, Ab), where P = (r, 9, b) is the primary part and O = (Ar, A9, Ab) is the resultant part. Since we want to generate uniformly distributed training patterns over the whole quantized input space, the primary parts of training sample pairs can be trivially decided by adopting each quantization region in the input space as the primary part of a training pair. As illustrated in Fig. 6, to decide the corresponding resultant part of a training pair, the first step is to compute the similarity between the primary part of training sample pair and each cluster. The second step is to check if the similarity value can pass a threshold test for each cluster. The third step is to evaluate the desired resultant part of the training sample pair. Basically, the resultant part is generated by the weighted mean approach for the resultant parts of the mean vectors of all clusters passing the threshold test. Assume the mean vector of the jth cluster is denoted as CMj, where CM~ is expressed as the
62
J.-X Ker et al. , Fuzzy Sets" and Systems 91 (1997) 53 68
primary part of the clustered mean vectors
m
resultant part of the clustered mean vectors
:
:
2
~
Iu,,ctioo [I l [operotorjL,uoctlon,
,-t(-Mean
~
Weighted
~
(5)
primary part of sample pattern
(6)
interpolated training sample pair (pJ,...,p',...,p"[ ot,...,o',...,o')
(P",...,P',.-.,V')
Fig. 6. The sampling flow of interpolation phase.
Step 4: Calculate the resultant part, O, of the train-
following form:
ing sample pair; the weighted mean approach is applied, as expressed in Eq. (17):
CM.i = (piloi) = (mp) .... , mpii ..... m p i l m o l ..... moii ..... moil, where u and v denote the number of input/output dimensions, respectively. The whole interpolation procedure to generate the training sample pair (PtO) for each quantized region is explained as follows: Step
1: Evaluate the corresponding similarity measure, s~, between the primary part of training sample pair, P, and the mean vector of each cluster C j on each input dimension i by Eq. (14):
sj =
t~act-0rl
(14)
Step 2: Decide the total similarity between P and the mean vector of each cluster Cj by Eq. (15), tsj =
15)
Sj. i-1
o' = 5~.~=t TH(tsj) × ,s~× mo~ 25-1TH(tsfl x s~ f o r i = 1 to v.
(17)
Step 5: Repeat Steps 1-4 until all quantized regions are explored.
4. Experimental results Two experiments, function approximation of sinusoid functions and color calibration, will be discussed in this section. In the former experiment, the effects of standard CMAC, BMAC (higherorder B-spline CMAC), and FCMAC on various conditions are compared. In both experiments, the simulation process is divided into learning phase and recall phase. During the learning phase, the total absolute e r r o r , ~trainingpatt . . . . HO~ - Or tl, is defined as the error function. The training pattern sampling procedure in last section is invoked as the preprocessing task for the learning phase.
Step 3: Perform the threshold test for the similarity 4.1. Function approximation for sinusoid fimctions
degree of each cluster C; by Eq. (16t, TH(tsi) = •
10 if tsi/> OK, otherwise.
(16)
In this experiment, two sinusoid functions,
y = sin(nx/180) and z = sin(nx/180)sin(ny/180), 0 <~x,
J.-S. Ker et al. / F u z ~ Sets and Systems 91 (1997) 53-68 1.00--
1,00
050-
0.50 .
/
000-
-050.
-1 O0
I
'
I
1 O0
200
'
I
63
--
/
0.00 -
-050 .
X !
'
300
-1.00
400
,
0
,
1O0
(a)
,
2 O0
|
3 O0
X ,
! 400
(b) 1 O0
oq/ \ 000.
0.50
/
.
0.00
-050
X I
I
I
1 O0
200
300
(c)
! 400
x
-1 O0 0
100
200
300
400
(d)
Fig. 7. The approximated curves of sinusoid function by CMAC, BMAC, and FCMAC.
y ~< 360, are approximated by CMAC, BMAC, and F C M A C models. In this experiment, each input dimension is quantized into 20 intervals, and each quantization region is assigned with a training sample pair. Fig. 7 shows the output curves ofy = sin(rtx/180) approximated by CMAC, BMAC, and F C M A C models. The configuration for CMAC is c = 4; for BMAC is n = G = 4; and for F C M A C is r = 2 with fully connected addressing scheme. Fully connected addressing scheme is employed in both BMAC and FCMAC model, so, NL = 4" and 5", respectively. Obviously, the output curve of CMAC in Fig. 7(a) is staircased, which is far away from the desired path. The output curve approximated by BMAC, as shown in Fig. 7(b), is almost the same as the
desired curve. Figs. 7(c) and (d) show the output curves approximated by FCMAC with fl = 0.5 and fl = 0.8, respectively. We can see that the greater value of fl makes the smoother output result. Obviously, FCMAC and BMAC models are much better than standard CMAC model in function approximation. Fig. 8 illustrates the absolute error curves for y = sin(nx/180) approximated by CMAC, FCMAC, and BMAC with learning rate r / = 1.0 and 0.8. A training iteration means that all training samples have updated the fuzzy weights one time. It is obvious that FCMAC has good convergence rate than the other two. Finally, the effects on the convergence speed with respect to different neighborhood parameters (c of
64
J.-S. K e r et al. i/ Fuzzy Sets and Systems 91 (1997) 5 3 - 6 8
16.00 -I total [ absolute 1 ~ error 12.00
s oo
11=10
m
19
I ~ 4,00 I
B
~
~
............
~
16.00 ~ total / absolute error 12.00
tMA~:
I 4.00
~. ~
0
_-_7-_- ~.S~
800
I,LMAt
I,..\ \ \
0.00
no. of iterations 4
rl=0.8
8
12
16
'f{ ~,~
I
r
0
4
000
20
FCMAC
"
t
I
8
12
-
(a)
110 o f
I 16
iterations I 20
(b)
Fig. 8. The absolute error curves for the sinusoid function approximated by CMAC, BMAC, and FCMAC.
100 -
no. of iterations /\
80
\
/
\
/
60
CMAC /
. . . . . . .
/
40_
• -
BMAC FCMAC
I!
I
tt
I I
20
" "
t
!
/
l~=0.8
,, . . . . . . .
,, "
CMAC: c BMAC: (; FCMAC : r
""
f~=0.7
/
0
'
0
I
'
4
I
8
'
I
size of neighborhood
12
Fig. 9. The stability curves for CMAC, BMAC, and FCMAC.
C M A C , G of B M A C , and r of F C M A C , respectively) are evaluated. The termination condition for each model is in the iteration that the absolute error of each training sample is smaller than a given threshold. Fig. 9 illustrates the statistic results with 0.01 threshold. Obviously, F C M A C model exhibits better stability than the other two. The simulation results for approximating the twoinput sinusoid function, z = sin(rtx/180) sin(~y/180), by C M A C , B M A C , and F C M A C models indicate
that the same conclusions can be reached. F C M A C exhibits better stability and convergence speed than the other two models. Fig. 10 illustrates the individual output responses for these three models. 4.2. Color calibration
As shown in Fig. 11, the operation flow of a color reproduction system can be briefly described as follows. First, the original color pictures are digitized
65
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
(a)
(b)
.:z
,,.
(c) "
K'X
•
~.)C
Desire output : z - sm(1-~6). sm(1-~) CMAC : q=20, c=4 BMAC : q=20, n=4, G=4, NL=I6 FCMAC : q=20, r=2, NL=25 number of training patterns: 400 Fig. 10. The approximated output of a two-input sinusoid function by CMAC, BMAC, and FCMAC.
Original ~ color image (o)
FCMAC [i
I Printed color calibration]~ printer ]-~ image (P)
co,o, i
Fig. 11. The role of color calibration in a color reproduction system.
by a scanning device and stored as image files in either hard-disk or CD-ROM. Then the digitized image files can be processed with desired image processing operations. Finally, to obtain real duplications of the processed image files, these image files have to be printed out by color printing devices. Due to the nonlinear mapping relationship between the scanning and printing devices, we need a color calibration mechanism to make the colors in reproduced image as identical as those of original one. In this experiment, F C M A C is adopted as the color calibration mechanism. In our experiment, the scanning device is the UMAX UC300
color scanner, and the printing device is the F A R G O PrimeraPro thermal color printer. There are two major parameters, 0s and 2, to be determined before F C M A C can properly calibrate color images. We can see that the image with 0s = 1.0 is almost the same as the uncalibrated image. The reason is described as follows. To have a nonzero resultant part (i.e., nonzero color calibration amount), the training sample pairs generated by the proposed sampling procedure with 0s = 1.0 must be very similar to one of the raw training patterns. Obviously, in this case, there are very few sample patterns satisfying this requirement. As
J.-S. Ker et al. / Fuzzy Sets and S.vstems 91 (1997) 53-68
66 80.00
]RMSE I ;~=-0.9
60.00
X=-0.5
L=O.O
X=2.0
40.00
~L=I0.0 20.00
gs 0.00 0.00
0.20
0.40
0.60
0.80
1.00
Fig. 12. The R M S E curves with respect to different 0~ and ,L
a consequence, very few pixels in the reproduced image can be calibrated, so that the calibrated image looks like the uncalibrated one. On the contrary, in the case of 0s = 0, almost all the clusters can pass the threshold test to take part in the decision of the resultant part of a training sample pair. In other words, each training sample pair will be affected by most raw training patterns in generating the resultant part. Therefore, the calibrated image may appear improper calibration effect due to overlearning. Fig. 12 illustrates the curves of root mean square error (RMSE) with respect to different settings of 05 and 2. Finally, we adopt 05 = 0.9 and ). = 2.0 as the parameters to extract training samples. The FCMAC model is configured with three inputs, and three outputs; the quantization resolution is set to 10 on each input dimension, and the learning rate q is 0.5. Fig. 13 shows the RMSE curve for training with color calibration training sample pairs. It indicates that FCMAC has a fast convergence speed in adjusting the fuzzy weights to approximate the nonlinear relationship between scanning and printing devices. Fig. 14 shows two experimental images
to illustrate the capability of FCMAC in color calibration. Figs. 14(a) and (b) present the reproduced images without applying any color calibration scheme. They do not look like the original pictures. Figs. 14(c) and (d) illustrate the reproduced images when FCMAC is invoked. Obviously, the colors presented in these calibrated images are more vivid and realistic.
5. Conclusions
FCMAC can be applied to many application fields to approximate the nonlinear control function of target systems. It is based on a local weight updating scheme, which has several advantages than the global weight updating scheme used by backpropagation neural networks, such as the features of fast learning and insensitivity to the order of training patterns. The information stored in the weight memory of FCMAC are fuzzy numbers instead of crisp values, so it stores fuzzy associations with distributed encoding, and manipulates the stored information by using fuzzy set theory. Based
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53-68
67
RMSE
6o.oo
60.00
40.00
20.00
no. of iterations o.oo
I
0
4
'
8
I '-
12
'
I
'
16
I
20
Fig. 13. The convergence curve of F C M A C during color calibration learning.
(a) Image 1 - reproduced w i t h o u t calibration
(b) Image 2 - reproduced w i t h o u t calibration
(c) Image 1 - calibrated by FCMAC
(d) Image 2 - calibrated by FCMAC
Fig. 14. Calibration results by FCMAC. For a color version of this figure see URL: http://www.elsevier.com/locate/fss
68
J.-S. Ker et al. / Fuzzy Sets and Systems 91 (1997) 53 68
on the B M A C architecture, F C M A C blindly extracts a n d refines fuzzy rules from t r a i n i n g samples. F r o m the s i m u l a t i o n results of a p p r o x i m a t i n g sinusoid functions, we can k n o w that F C M A C has a good convergence rate a n d the o u t p u t plane of the controlled plant is s m o o t h e r than C M A C model. Besides, the optimal values of the neighborhood p a r a m e t e r c of C M A C a n d G of B M A C are hard to be selected, but the F C M A C has a good stability on the radius p a r a m e t e r r. This can reduce the design effort of F C M A C . F u r t h e r m o r e , the bell-shaped receptive field functions in F C M A C is easier to be m a n i p u l a t e d than the recurrent Bspline receptive field functions in BMAC. F o r the color calibration experiment, the parameters of 0s a n d ,:. in the s a m p l i n g procedure will affect the calibration results. A higher value of 0s will cause the lower calibration efficiency. The p a r a m e t e r 2 presents similar effect on color calibration as 0,. F r o m the visual perception of the calibrated images, we can conclude that F C M A C can result in excellent color r e p r o d u c t i o n on color images.
References [1] J.S. Albus, A theory of cerebellar function, Math. Biosci. 10 (1971) 25-61. [2] J.S. Albus, A new approach to manipulator control: the cerebellar model articulation controller, J. Dyn. System Measurement Control 97 (1975) 220-227. [3] J.S. Albus, Data storage in the CMAC, J. Dyn. System Measurement Control 97 (1975) 228 233. [4] J.S. Albus, Mechanisms of planning and problem solving in the brain, Math. Biosci. 45 (1979) 247 293. [5] J.S. A l b u s , B r a i n s , Behavior and Robotics (BYTE/McGraw-Hilt, Peterborough, NH, 1981) 143 186. [6] M. Braae and D.A. Rutherford, Fuzzy relations in a control setting, Kybernetes 7 (1978) 185-188. [7] C.W. Chang and P.R. Chang, Neural plant inversecontrol approach to color error reduction for scanner and printer, Proc. Internat. Conf. on Neural Networks (1993) 1979-1983. [8] P.J. Hartley, Tensor product approximations to data defined on rectangular meshes in N-space, Comput. J. 19 (1974) 348-351.
[9] D. Hubel and T.N. Wiesel, Receptive fields, binocular interaction and functional architecture in cat's visual cortex, J. Physiol. 160 (1962). [10] H.R. Kang and PG. Anderson, Neural network application to the color scanner and printer calibrations, J. Electron. Imaging I (1992) 125-135. [11] J.S. Ker, Y.H. Kuo and B.D. Liu, Hardware realization of higher-order CMAC model for color calibration, Proc. IEEE lnternat. Conj'. on Neural Networks (Perth, Australia, 1995) 1656-1661. [12] A. Kolcz and N.M. Allinson, Application of the CMAC input encoding scheme in the N-tuple approximation network, lEE Proc. Comput. Digital Techniques 141 0994) 177 183. [13] Y. Linde, A. Buzo and R.M. Gray, An algorithm for vector quantizer design, IEEE Trans. Commun. COM-28 (1980) 84 95. [14] E.H. Mamdani and S. Assilian,An experiment in linguistic synthesis with a fuzzy logic controller, lnternat. J. ManMachine Stud. 7 (1975) 1-13. El5] S. Manglevhedakar, An adaptive hierarchical model for computer vision, Thesis, Louisiana State Univ. (1986). [16] W.T. Miller, R.P. Hewes, F.H. Glanz and L.G. Kraft, Real-time dynamic control of an industrial manipulator using a neural-network-based learning controller, IEEE Trans. Robot. Automat. 6 (1990) 1--9. [17] J. Moody and C, Darken, Learning with localized receptive fields, Proc. Connectionist Model Summer School (1988). [18] T. Poggio and F. Girosi, Networks for approximation and learning, Proc. IEEE 78 (1990) 1481-1496. [19] J.C. Smith, D.C, Tribolet, H.E. Mostafa and E. Maghakian, Development of color graphics printer, IEEE Trans. Consumer Electron. 34 (1988) 523 529. [20] D.L. Sparks and J.S. Nelson, Sensory and motor maps in the mammalian superior colliculus, Trends Neurosci. 10 (1987) 312 317. [21] G.K. Starkweather, A color-correction scheme for color electric printers, Color Res. Appl. (Suppl.) 11 (1994) 108 114. [22] H.L. Stephen, A.H. David and J.G. Jack, Theory and development of higher-order CMAC neural networks, IEEE Control Systems 12 (2) (1992) 23-30. [23] S. Suzuki, T. Kusunoki and M. Mori, Color characteristic design for color scanners, Appl. Optics 29 (34) (1990) 5187-5192. [24] P.Z. Wang and H.M. Zhang, Truth-valued flow inference and its dynamic analysis, Proc. Peking Normal University, 1 (1989).