- Email: [email protected]
The design algorithms of membership functions for a fuzzy neuron
ZZY
sets and systems ELSEVIER
Fuzzy Sets and Systems 71 (1995) 329-343
The design algorithms of membership functions for a fuzzy neuron M a s u o F u r u k a w a a'*, Takeshi Y a m a k a w a b =Department of Electrical Engineering, Nagano National College of Technology, Nagano, Japan bComputer Science & Control Engineering, Kyushu Institute of Technology, Fukuoka, Japan
Abstract This paper describes two design algorithms of membership functions for a fuzzy neuron using example-based learning with optimization of allocation of cross-detecting lines. One algorithm is called the elimination method which eliminates the inefficient cross-detecting lines and the other is called the selection method which picks up the efficient cross-detecting lines. The features of these algorithms are that no knowledge of experts is required to design, and the least cross-detecting lines are arranged. We verified these algorithms by recognition of hand-written characters. In comparison with the elimination method, the selection method can reduce the number of cross-detecting lines common to all figures (i.e. the number of the sensor arrays) and the CPU time for design of the membership functions of a fuzzy neuron. Keywords: Fuzzy neuron; Pattern recognition; Design of membership function
1. Introduction The artificial neural networks I-5, 8] employing conventional neuron model or analog threshold element are not designable, although each neuron model is very simple. To cope with this problem, a f u z z y neuron has been proposed [12-1 and applied to a high speed recognition hardware system of hand-written characters 1-10], in which the weighting factors and summation of the mathematical model of neuron are modified by membership functions and M I N operation, respectively and a threshold level of neuron is not assigned. Its features are easy to design and robust against the noises and defects in the pattern. The fuzzy neuron has been implemented in a silicon chip called a f u z z y neuron chip. The fuzzy neuron chip is of very high speed and compact in addition to those features. On the other hand, various methods of character recognition have been developed 1-1,2, 4, 6, 7, 9]. Though most of those methods are too complicated to implement in hardware, the fuzzy neuron is an easy one. Among those methods, the zonde method [1, 2] seems to be similar to recognition by the fuzzy neuron. The advanced distinctions of the fuzzy neuron from the zonde method are no regulation for writing characters and more flexibility by membership functions.
* Corresponding author. 0165-0114/95/$09.50 © 1995 - Elsevier Science B.V. All rights reserved SSDI 0165-01 14(94)00278-9
330
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
Though a membership function makes the fuzzy neuron flexible and designable, know-hows of human experts are necessary for this design. In some applications, we cannot get the know-hows from a human expert or there is no human expert to do so. In those cases, a design algorithm without know-hows of human experts is needed. We proposed two design algorithms of membership functions for the fuzzy neuron using example-based learning with optimization of cross-detecting lines I-3, 11]. These algorithms facilitate system design without the know-how of experts. One of these algorithms is called the elimination method which eliminates the inefficient cross-detecting lines, and the other is called the selection method which picks up the efficient cross-detecting lines. Both algorithms are divided into three parts which are classification of pattern samples, assignment of membership functions and optimization of allocation of cross-detecting lines. Procedures of these three parts are summarized as follows. First, pattern samples of each categories for example-based learning are classified to adequate classes by their outlines. Second, a fuzzy neuron is assigned to each classes and membership functions for each fuzzy neuron are assigned by example-based learning from the cross-points which are provided by classified pattern samples. Thirdly, the optimization of the number and location of crossdetecting lines is achieved. While the procedures of the classification and the assignment are the same in both of those methods, the optimization procedure is distinct from each. The elimination method gives optimum cross-detecting lines by omitting inefficient ones. On the other hand, the selection method gives optimum cross-detecting lines by selecting out efficient ones. In this paper, both of these methods are discussed, implemented by software of EWS, verified by recognition of hand-written numerical characters and compared with each other concerning assigned membership function and CPU time.
2. A fuzzy neuron and its application for pattern recognition of hand-written characters 2.1. A f u z z y neuron
Fig. 1 shows a schematic model of a fuzzy neuron, in which X1,X2, ... ,Xi, ... ,X, are vectors of crisp sets and/or fuzzy sets to inputs, wl, w2. . . . . wi, ..., w, are membership functions of the synaptic weight and Y is an output of the fuzzy neuron ranging from 0 through 1. Excitatory and inhibitory connections give the grade of soft matching/~ between the input Xi and the synaptic weight wi. The grade #i is fed to a MIN block of the fuzzy neuron model. Excitatory connections are represented by MIN, and inhibitory connections are represented by fuzzy logic complements followed by MIN. The output of the fuzzy neuron is a minimum
W,
, y
Fig. 1. A schematic model of a fuzzy neuron.
M Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
331
grade of connections and gives the grade of soft matching between all of inputs and all of membership functions. 2.2. Pattern recognition of hand-written characters by a fuzzy neuron A fuzzy neuron is able to recognize one character by itself. Pattern recognition of hand-written characters by a fuzzy neuron has been discussed [10], in which cross-detecting lines are employed to get features of characters as shown in Fig. 2. There are three regions on the cross-detecting line, a crossing region, a forbidden region and a don't care region. The crossing region is the portion of the cross-detecting line where the character should cross. The forbidden region is the portion of the cross-detecting line where the character should not cross. The don't care region is the portion of the cross-detecting line where the character will uncertainly cross. One or more cross-detecting lines are assigned in the frame. The cross-detecting lines of all characters, which are put into the same input frame, are referred to common cross-detecting lines. The boundaries of these three regions are fuzzy, so that they should be defined by membership functions for each cross-detecting line as shown in Fig. 3(a). If there are two or more crossing regions on one cross-detecting line, the membership function assigned on that cross-detecting line should be divided into
c.o,o . oo. 7
._SOT
A t ~ "-..-'-..~.i ............................ --
,' .-"
B
. oo.
f . .............. ......,
".. ............ iy"
q
%..
pl
i
_z
(a) 1I 0 #2
$
W U) O n.
CROSSING REGION
o
FORBIDDEN R "GION
" i
~
c!i 'o'oo "'ro" /i )J
#3
C
/
(c) 11 0 III
T il
!a
I I I I I I I I I I I I
[b
lle I I I I I I
~c
liB I ~
CROSSPOI CROSSING REGIONJ CROSS-DETECTINGLINE
1
Fig. 2. Cross-detecting line in the frame.
SENSORARRAYIN\ CROSS-DETECTINGLINE ~ Fig. 3. Assignment of the membership function of a crossing region.
332
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
FORBIDDENREGIO~'-~~ A R E
BREGION'~O~BIDDEN REGION "7
/~4
i
: i
0 F5
!d
'
!b
w z II¢3 '"
ie
i7
CROSSPOINTS SENSORARRAYIN CROSS-DETECTINGLINE
Fig. 4. Assignment of the membership function of a forbidden region.
two or more membership functions as shown in Fig. 3(b) and (c) in order that a cross-point of the don't care region does not affect the results. These membership functions are used for excitatory connections. Fig. 4 shows a membership function of two forbidden regions. The membership function of forbidden regions are used for inhibitory connections. The don't care region is not used for recognition of characters, because it has no information. Membership functions for crossing regions and forbidden regions can be assigned on a cross-detecting line. The appropriate number of cross-detecting lines are located horizontally and vertically in the frame as common cross-detecting lines. Cross-points detected by sensor arrays arranged on common cross-detecting lines are applied to input buses of a fuzzy neuron. An output of each fuzzy neuron represents the grade of the possibility of the character.
3. Automatic design algorithm of membership functions using example-based learning with optimization of cross-detecting lines Fig. 5 shows design flow of membership functions using example-based learning with optimization of cross-detecting lines. In example-based learning, when the variance of pattern samples is big, the feature of the patterns defuses and is very difficult to squeeze out. To cope with this problem, each category of pattern samples should be classified to adequate classes. And a fuzzy neuron is assigned to each classes. In other words, the number of fuzzy neurons needed to achieve a reasonable pattern recognition is equal to that of whole classes. Thus membership functions for a fuzzy neuron should be assigned from each class.
M. Furukawa, T. YamakawaI Fuzzy Sets and Systems 71 (1995) 329-343 CATEGORY "0"
CATEGORY"1"
333
CATEGORY "9"
i 0 i 3, iiiiiii!ii iiiiiiiilililili.......... PATTERN SAMPLES
~ ..o ~'"'"'I
~--.r
.......
• .. INITIALi..~
PAI"rERN SAMPLES
PATTERN SAMPLES
.......l .......
~
I
I ~'~'~'~'°"
I
~................ ,"
.......1........ -I
........
~ --- ~
. . . 1
i'"i ~t~'~s ~ ~,J,cno,s ~"'i ~,er~
i ..........
~ ~J,c.~o,s i"" i FU,e~....S....
OPTIMIZATIONOFALLOCATIONOF CROSS-DETECTINGLINES
OPTIMUM MEMBERSHIP FUNCTIONS (Cluster Z1)
MEMBERSHIP FUNCTIONS (ClusterZ2)
• • • • • • • • •
MEMBERSHIP FUNCTIONS (Cluster Nn 9)
Fig. 5. Design flow of membership functionsfor a fuzzy neuron. Pattern samples of each class give membership functions in all the cross-detecting lines on the each frame. Membership functions are assigned from cross-points of a cross-detecting line and pattern sample. A frame carries many cross-detecting lines and membership functions are assigned on all cross-detecting lines. It means that the fuzzy neuron has the same amount of synaptic membership functions or more. However, all of the membership functions are not necessary for a reasonable pattern recognition depending upon the complexity of the pattern and the number of the categories to be distinguished. Thus, thirdly, the optimization of the number and location of cross-detecting lines is achieved. 3.1. Assignment of membership functions using example-based learning 3.1.1. Classification of pattern samples in a category Pattern samples can be classified by a human at a glance or by the computer which executes the following algorithm.
334
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329 343
The cross-detecting lines are assigned in a frame to classify pattern samples. In this paper, 64 horizontal cross-detecting lines and 64 vertical cross-detecting lines are assigned to classify pattern samples. The points of a character on the cross-detecting line, which are crisp, are fuzzified into a trapezoidal membership function to compare rough outlines of a character. In order to define the shape of this membership function, let L represent a width or a height of the frame. represents a fuzzification coefficient which takes the value ranging from 0 to 1.0. t represents a width of a character crossing to the detecting line. Then the support of the membership function will be ~L + t, as shown in Fig. 6. After all pattern samples are fuzzified, the pattern samples in a category are classified into some classes by a similarity measure Sij between the ith and the jth fuzzified pattern samples. A similarity measure Sij is defined as [ ~e'= I ]Ae'l'i A ]Ae,l,j 1 ~ e = 1 [2eli A [~elj 71SiJ -- g -4-~ s 1 / = 1 ~u= 1 12eli v lieu l'= ~ eU''= 1 12e'l'i V ~te,l, j J
,
(1)
where lleti represents the eth element of a sampled membership function on the/th horizontal cross-detecting line in the ith pattern sample, U represents the number of elements in the horizontal universe of discourse, or the number of horizontal cross-detecting lines in the class. The ' represents vertical ones. Step 1: The first pattern sample is regarded as a reference and Slj is calculated for all j's except this reference. Step 2: The first pattern sample and the others which satisfy the condition Slj > [Sd]k,
should be included in one class, where [Sa] k represents the threshold similarity measure for discrimination in the kth class. Usually [Sd]k'S for all classes are equal to each other, that is, [Sd]k = Sa. Step 3: The residual pattern samples are renumbered. Step 4: Repeat Steps 1-3 until the classification is completed.
L
0
/\ i o,' Li iaLi i "2-.i~ t.i~ 2~.i
Fig. 6. Membershipfunction to fuzzifya pattern for clustering.
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
335
A human can classify by comparison the features of the patterns at a glance. The features which are employed by human to classify, are line width of the pattern, curve rate of the line, tilt of the pattern and size of the pattern for example. The result of the proposed classification is close to that of what the human classifies, if • and Sd are suitable values. The similarity between rough outlines of patterns in the same class is roughly proportional to Sd and reciprocal ~ values. The values of ~ and Sd are assigned under the following conditions: (a) The number of classes is close to that of what a human classifies. (b) The rough outline of patterns are similar to each other in the same class. (c) ct is assigned for each membership function not to overlap with neighbor one on the same cross-detecting lines.
3.1.2. Assignment of membership functions of a fuzzy neuron Classified patterns will provide us with cross-points on horizontal and vertical cross-detecting lines corresponding to the class of pattern. The membership functions for a crossing region and a forbidden region can be assigned from these cross-points. Fig. 7 (a) shows, for example, only one (vertical) cross-detecting line with cross-points produced by three samples of hand-written character "3". The lines where membership functions are obtained by this assignment procedure are called initial cross-detecting lines, because they should be optimized. First, in order to assign the membership function for a crossing region, these cross-points are clustered by a simple method with the cluster radius rd as follows: Step 1: The first data (positions of cross-points, al,a2,a3,a4) of character in a class is located on a cross-detecting line. It is examined that each point is located inside of clusters, the center of which is al, a2, a3 or a4. If each cross-point is not included in each cluster (independent from each other), all of these cross-points can be centers of clusters. If some cross-points are included in other clusters, these cross-points should be regarded to be in one cluster and a virtual center of this cluster is a mean position of these cross-points. In case of Fig. 7, for instance, a2, a3 are in the same cluster, the center of which is a midpoint of a2 and a3 on a cross-detecting line. Step 2: It is examined that the next data (positions of cross-points, bl, b2, b3) of character in the class are included in the clusters of radius rd defined in Step 1. In the similar manner, all the data are examined to be included in the clusters. Step 3: The clusters should be classified to a crossing region or, a don't care region. If all the pattern samples give one or more cross-points to the cluster, it is a crossing region. Otherwise, it is a don't care region. In case of Fig. 7, cross-points are classified to three clusters (regions). The first cluster (al,bl, cl) and the third cluster (a4, b3, c2) include all the pattern sample, so that these region are crossing region. The second cluster (a2, a3, b2) does not include pattern sample "c", so that this cluster (region) is a don't care region. Step 4: A membership function of crossing region is assigned as follows. The outer elements (al and bl, and b3 and c2 in Fig. 7) of cross-points in the crossing region can be the shoulders and give the spreads outside by ~L/2 as shown in Fig. 7(b), where t represents the distance between outer elements of cross-points, L represents the length or width of the frame, • represents fuzzification coefficient. Thus the support of the membership function is ~L + t. There is no need to assign the membership functions for don't care regions, because they are not used for recognition. The forbidden regions can be assigned only if the distance between shoulders of neighboring crossing regions are greater than 2fl + ~L as shown in Fig. 8. In this figure fl is a constant representing a distance between the spread of the forbidden region and the shoulder of neighboring crossing region. The spreads of the forbidden region are the points outer the shoulders by ~L/2 as those of the crossing regions.
336
M. Furukawa, T. Yamakawa/ Fuzzy Sets and Systems 71 (1995) 329 343
(a)
- - CROSS-DETECTING LINE
a R .................................
. . . . . . . . . .
bl cl a2 a3 b2 b3 a4 c2-
ILL;
1I.---.~.
, (b) DON'T CARE REGION CROSSING R E G I O N ~ CROSSING REGION al blcl a2 a3b2 b3 a4c2
~--T
-
--
i
T--T
o
\ iaLi
f~ a"L+t ,:'~ Fig. 7. Assignmentof membershipfunctionfor crossing regions. (a) Cross-points between character "3" and a cross-detecting line. (b) Membershipfunction for crossing regions.
-
aL 2
iaL 2 i
i
GL~ 21
,K 9 Fig. 8. Assignment of a membership function for a forbidden region (solid line).
3.2. Optimum allocation of cross-detecting lines A frame carries many cross-detecting lines (typically 64 x 2 = 128 lines). It means that the fuzzy neuron has the same amount of synaptic membership functions which are obtained by the procedure described in Section 3.1.2. However, all the membership functions are not necessary for a reasonable pattern recognition depending upon the complexity of the pattern and the number of the categories to be distinguished. Thus it is possible to reduce the cross-detecting lines, i.e. membership functions for the efficient recognition. The procedure for this optimization of allocation of cross-detecting lines is discussed in this section. In this paper, the number of horizontal cross-detecting line and vertical cross-detecting line are 15 and 15 for saving C P U time, respectively. (However, in Section 3.1.1, 64 cross-detecting lines are assigned for each direction, to classify pattern samples with similarity measure.) These cross-detecting lines are assigned in a frame uniformly, as shown in Fig. 9. This optimization is carried out by different procedure for the elimination method and the selection method.
3.2.1. Optimization by the elimination method To explain optimization by the elimination method, following criterion for eliminating of cross-detecting line is presented. Criterion for eliminating cross-detecting line (CEL): The following notation is assigned. X(k): a pattern or a character X of the kth class which should be recognized; xdk): the ith pattern sample in the kth class X(k), i.e. xi(k) ~ X(k); FNX(k): a fuzzy neuron designed compatible to a class of character X(k); OUT(X(k); xi(k)): an output of F N X ( k ) when xi(k) is input;
M. Furukawa, "1~ Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343 VIV273
•
• 77
337
V15
H1 H2 H3
H7
H15
Fig. 9. A s s i g n m e n t of cross-detecting line.
Y(/):
a pattern or a character Y of the/th class which is out of the category including a class X(k); y j(1): the jth pattern sample in a class Y(I), i.e. yj(l)E Y(I); OUT(X(k); yj(/)): an output of FNX(k) when yj(l) is input; d: a minimum distance between OUT(X(k); xi(k)) and OUT(X(k); yfl)) (typically 0.9). The criterion: If the cross-detecting line under the test is omitted and A OUT(X(k); x,(k)) - V V V OUT(X(k); yj(/)) >~ d, i
r
l
(2)
j
then the cross-detecting line should be omitted. Otherwise, the cross-detecting line should be held. Checking of cross-detecting lines in case of Fig. 9 is achieved in the following sequence using the CEL. If the number of cross-detecting line is changed, the order of checking of cross-detecting lines should be determined in the similar manner. Step 1: The horizontal cross-detecting line of the center of the frame is examined by the CEL. Step 2: The vertical cross-detecting line of the center of the frame is examined by the CEL. Step 3: The horizontal cross-detecting line of the center of the upper half of the frame is examined by the CEL. Step 4: The vertical cross-detecting line of the center of the left half of the frame is examined by the CEL. Step 5: The horizontal cross-detecting line of the center of the lower half of the frame is examined by the CEL. Step 6: The vertical cross-detecting line of the center of the right half of the frame is examined by the CEL. Step 7: The horizontal cross-detecting line of the center of the top quarter of the frame is examined by the CEL. Step 8: The vertical cross-detecting line of the center of the most left quarter of the frame is examined by the CEL.
Step 29:" The horizontal cross-detecting line of the center of the bottom eighth of the frame is examined by the CEL.
Step 30: The vertical cross-detecting line of the center of the most right eighth of the frame is examined by the CEL.
338
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
3.2.2. Optimization by the selection method The selection method is achieved by two procedures. First, the procedure for optimization of common cross-detecting lines is carried out. In this procedure, common cross-detecting lines are selected out from initial cross-detecting lines as efficient cross-detecting lines for recognition. Second, the procedure to obtain optimum allocation of cross-detecting lines of each fuzzy neuron is carried out. In this procedure, efficient cross-detecting lines for each fuzzy neuron are selected out from common cross-detecting lines. To explain these procedures, the following notation is assigned. CDL(mi.it): the minitth initial cross-detecting line; CDL(m~om): the mcomth common cross-detecting line; X(k): a pattern or a character X of the kth class which should be recognized; xi(k): the ith pattern sample in the kth class X(k), i.e. xi(k)e X(k); CDL(X (k), mopt): the moptth optimum cross-detecting line for the kth class X; FN X(k) CDL(minit): a fuzzy neuron using only one cross-detecting line CDL(mi.it) of a class of character X(k); OUT(X(k) CDL(mi,it); xi(k) ): an output of FN X(k) CDL(minit) when xi(k) is applied to the input; Y(/): a pattern or a character Y of the /th class which is out of the category including a class X(k); thejth pattern sample in a class Y(l), i.e. yj(l)e Y(I); Yfl): OUT(X(k)CDL(m~.it); yj(l)): an output of FNX(k) CDL(miait) when yj(l) is applied to the input; FN X(k): a fuzzy neuron designed compatible to a class of character X(k); d': a minimum difference between OUT(X(k)CDL(mi,it); xi(k)) and OUT(X(k)CDL(mi.it); yj(l)) (typically 0.9); NUM(X (k)CDL(mi,it); Y(/)): the number of the classes Y(l) including pattern samples which are not accepted as a character X(k) by FN X(k)CDL(mi.it), i.e. the number of the classes Y(l) which satisfy
A OUT(X(k)CDL(m~°~t); x,(k)) - V OUT(X (k)CDL(m~.it); y fl)) >~d' ; i
(3)
j
NUM(X(k)CDL(m¢om); Y(I)): the number of the classes Y(l) including pattern samples which are not accepted as a character X(k) by FN X(k)CDL(m¢o,n). i.e. the number of the classes Y(l) which satisfy AOUT(X(k)CDL(m¢o~); x,(k)) - VOUT(X(k)CDL(m¢om); y.i(1)) >t d'. i
(4)
j
The procedure for optimization of common cross-detecting lines is as follows: Step 1: Calculate the sum of NUM(X(k)CDL(ml.it); Y(1)) for all mi,it's by
N .....
=
~ ~ NUM(X(k)CDL(m~.~,); Y(l)).
(5)
X k
Step 2: Pick up the cross-detecting line CDL(minit) which has the maximum value of N ..... in initial cross-detecting lines as one of the CDL(mcom). Step 3: Discard the class Y(l), which satisfies the Eq. (4), from Y(l). Step 4: Repeat Step 1-Step 3 until Y(l) becomes empty. The procedure to obtain optimum allocation of cross-detecting lines of FN X(k) is described by the following steps: Step 1: Calculate the sum of NUM(X(k)CDL(mcom); Y(l)) for all mcom'Sby N .... = NUM(X(k)CDL(m~om); Y(l)).
(6)
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
339
Step 2: Pick up the cross-detecting line CDL(mcom) which has the maximum value of Nm°o. in all moore'S as one of the CDL(X(k), mopt). Step 3: Discard the class Y(1), which satisfies from Y(l), A OUT(X (k)CDL(X (k), mopt), xi(k) ) - V OUT(X (k)C DL(X (k), mopt), yj(l) ) >t d'. i
(7)
j
Step 4: Repeat Step 1-Step 3 until Y(l) becomes empty.
4. Experimental results
4.1. Optimum allocation of cross-detecting lines and assignment of membership functions The selection method and the elimination method are implemented by an EWS. The algorithm of pattern recognition is implemented by a PC. Patterns for recognition are ten figures which are from "0" through "9". The programming language is the C language. The data of patterns and membership functions can be transferred between EWS and PC each other. To verify the algorithms, membership functions of ten figures which have been assigned by example-based learning were optimized by both methods. Twenty pattern samples for each figure have been used and the pattern samples have been classified into 35 classes by their outline. Where 20 pattern samples were employed for one figure, sizes of the frame to input a pattern are 64 x 64 dots and ~t, fl, ra and d are 0.1875, 0, 10, and 0.9, respectively. In order to estimate the recognition rate for the patterns which are not employed to learn membership functions, we provided five pattern samples for each figure (i.e. 50 pattern samples for ten figures). Fig. 9 shows locations of horizontal and vertical cross-detecting lines which are assigned initially to optimize cross-detecting lines. There are 15 cross-detecting lines for horizontal and vertical before optimization. Then, the recognition of hand-written characters was carried out using the membership functions after optimization. Figs. 10 and 11 show locations of cross-detecting line for all classes of ten figures after optimization by the elimination method and the selection method, respectively. A few cross-detecting lines for horizontal and vertical are required to recognize the figure successfully. Obtained cross-detecting lines for each categories, shown in Fig. 10 for the elimination method and Fig. 11 for the selection method should be merged to the common cross-detecting lines as shown in Fig. 12(a) and (b). And sensor arrays should be located as shown in Fig. 12(a) or (b).
4.2. Comparison between two methods Fig. 13 shows the number of the cross-detecting lines of each fuzzy neuron after both optimizations. The selection method decreased the number of the cross-detecting lines of each fuzzy neuron in thirteen classes from it of the elimination method. But, in three classes, the selection method increased it. The number of the sensor arrays by the selection method was 38% less than the elimination method. The processing time for the design by the selection method was 1 h and 27 min and that of the elimination method was 3 h and 47 min. Thus, the processing time of the selection method was 62% shorter. Both of these methods can recognize the patterns which are employed to learn membership functions, completely. In the recognition experiment for the patterns which are not employed to learn membership functions, the recognition rate is 82% for the selection method and 86% for the elimination method.
340
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
Ii i il i. ii::ii. l L~:LL+ I-2L+L~:.r!±
I
;~:gi-"/i ~;Ti~:;i~.
;r-;;~;r-~;T ;:~i~ Forclass
~-~;÷; -~÷--~i" I
I ofafilu~e'0
"
r o t class 2 of • figure "0 "
p,il ii,ii ii,I 14-~. I # 14~bl-,b ,l-ll 4-
i .......... | .... : For elm I ofa filiure " I "
|~:~i$i :~i; l,":il; I;:IIT; il;~:;;~ For c l a m 2 o t ~ fisure " l "
,~-~,~-, HTH-.-T-
;iTiTl ;i~-;Ti~ I~:~I.T.I ~t~ ~:~t~Fo~clms
3 ofafiEure'l
"
Forclass 4 o f a f i s u r e " l
"
li:ll ll:ll?.ll. I~-
HIIIIIIJI II-~.I
For class [ oi" a fl~lure " 2 "
II.$1-1-II. |.1.
For class 2 o / a fil~Ure " 2 "
I..............[i; i~-'ii$;;T;~iT
L:fi ~--[~ i:':iii ff~_-:iiii: ~:~Zr I ol'a filu~e'3
"
i ........
r
ii:iiii~
ii:ii~-ii:i ,7 ~ i-Tii:il$11:i $ ~ ii
ii:liT;TI
F o r class 2 o f a fillure " 3 "
Forcl~s
fii::ii i ii li i:=:I P''T
i;i:.;17. Tii~i-T-
11:iIIi~, $ii.~ii 11:;1-7.;~ -?liT;;
IIT;7-IT; -T- ;T
I. I .I-I I + 14-I. I ~.I I-H~.-I Forclan
~:.:.::.~.: .-'~6.::-::L=
i ......
~o, ~.l~:.3..o.r:n.pff-~i/:ilZ-h":l ,~ ~ i-~
i :':l i -?.i ,~:.~~.T.i 1: i;
;T;i ig~;T TgiT
i;:~;;;]ii;;;;;z. I 14.11,¢.I -I 1-1-1-1-:-I-1-
~ofafillU~']"
Forcl~s
4ofafilu~'3"
For clus
S ofafilum
"3
fii::ii ilii i i. l
14.~.14-1.l.ll.~- I l . l l 4 -
I l.ll'l'l
11~'14"II+
liiii,ilii,ii, liT;T; ;iT;Ni,7
1.14:H..~ I ¢::-I-:- II:F i + I For c l a ~ I o i ,
Illume " 4 "
For class 2 o f
"4 "
s filiute
i::::::l::::::III
For class 3 o f I fi~ure " 4 "
Iiiiii it!i!! !! l¢.ll4-I
;TIII;TI;~.I~:;;~ i~:ii~.;l:li '~i~:~i~ l:orclm
14-~.I-:. 14-} l.l- I¢.~. I -:.
:i;:i;.~i;:~;T;i:;i$
I o f a failure " 5 "
For class 2 o f a f i l u r e " $ "
ll~.ll-ll-:-
hi ii ib:ii
i ...... | ........ For c l ~ a ~ o f a f i ~ v ~
i
"~ "
iiNiiii iiiI i
,.~:,.,.2 2 ,.2 ,.:,.,..+I For class I o t a f i l u r e " 6 "
For ¢l~s 2 ola fillwe "6 "
I
fiitiii i. l itl
i .......... i .... : Forcl~s ~ of a ilium'6
"
................. For e l m 4 o f o r q l u ~ ' 6
"
Iii li:~iiiil~ii~i~l I~ 14-I¢-~1¢-; 4-~1-;
i~ i$;Tii-Tii:llT Q,.I
For clm
. . . . . . . . . . . .
lli
il.'Ii-T; lIT ~iT
;;T;T; .......
i
1 o f a failure " 7 "
i,'.:~$i ;i$ ~i$
;;;;:;i.:: ........
For class 2 of a filure = 7 "
Fo~clul
] o~a f i l u r e " 7 "
J . : l - ~ I J - : ; 4 ; J - : 1 4 : J-
I-; ~-~1":-~I t ¢ 441-;; ~-I
IT;iT; l T ITIT
~ '~}4
~'}-H~,
-
I'I 4":' I ~" I ~l'll"el 1-~"I ":"
f!!!!!~!!~1.-'1~i,z For©Ira
I ota fillur~'ll
l "
~ ,He t4-t ;I.~
Fo~ class 2 o [ a fiSut¢ " 8 "
iTI bT.ii:;;..7;giir. i i i-'i; ~-;i:i i T; TI;r.. il:;;T;I:II~;TI;~ For c l ~ l l 3 a i m l i l u ~
"S "
i ~.%T.i |$i~:~|$ ; ~ i.T-; i.T-I;~;$ 14-$1 I '1.;.14-;,14.
, ......
F o r c l a s s I o f a fisur= " 9 "
For c l a .
.
....
.,..,
2 o f a fisure " 9 "
I" II'I" I "I' I l ' } l ' l " II'F I ~.'I F o r c l m s 3 oJ'a fillUre " 9 "
I" I'I'I" I I" I f l ' l + | 4 " l ' I'P Fol'chuls 4 o f a r l l u r e " 9 "
r..... l i ....... F o r claaa S o~r a fiSule " g "
: A cross-detecting line to require for recognize the fisure successfully. ...... : A omitted cross-detecting fine. --
Fig. 10. A l l o c a t i o n o f c r o s s - d e t e c t i n g lines after o p t i m i z a t i o n
: The frame line.
by the elimination method.
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
++m+l~+++,,m;:,=l i+m+.It.-+m+++++l
I+m~+.-I++I .+, ._+++++++I t!:~!±l.~:t I±.~,:t!± I-I'4-Fl-:- I 1.:-I'4- l 4-I" ":+3:11~. ,+,:i 2~i:~t2
r,+ll,,..:.+:,,
f,~+;.+H++.+I
Forclam
341
I of 41filure "O
For class 2 of 41 figure "0 "
Imtl;m+. p+-.H+tl:.-++++_+ ;;:;; ;;:.q-,7l+++tml ;:;i-'7. E+mI+i+i~im~l lim.;inlmml H+m. +l+;:;;Z.++++.:l ;;:;; if+iT13:11 i,+,:11+...,+.:i~?. 11:11 ~ i .:.:i:112 ~i~1~.1~12,3,12 I 1,3112,H,+I +;;+;+;~-ii~-;<-~i~-i H,++I +,+,,++, Iti.-+;It+It,mt:+l Iiitiii~i:.+:ii+i+l I~mlmm+,l i,+,:11 ;3:+1+i< i+ ;~;:r ;3:I;+;;:+;Z++++1111:++. :+,<.+-I+11
+++L+i+i;+i~i++:l
;~:;; ;~;-~;;-';;;
,<-.+,<-~ + . 1 1 1
;;.';;~,i ;i;;< 1;
Porc~.
For ¢ l • m 2 o/'41 figure " l "
L of 41fillUme'l
Forclan
$ofafigure'l
"
Forclms 4 o f a f i g u r e ' l
"
+:,.+,+++":.:q,~,:+, I++. +l++:++++++++l ~"+i!.?!.'+.!? !":':".? I+++ll!++l++l+tl,.-+.l ~;i:;;t;i:i;z.;~iz.I I++::++l+++}.-.+l+:+++l++++_+++++t+,++~++l +++++;:+-ml+l++++l+m;+.ml=.+tm+.l+;.:+:2Z'.-.IL+I,+.I,2,~.+,++ 13:11 13212 i:13.'+ill,+.' +++-.+l -+.+ll++l-.++l+<.+++++-++++l +-.ll+.l
~+.;:+.;l;,111.+,.;.+:i++:+I
;; i~-;;:;iZ i;-~i+-
1++ : +":+.~:++":~.':: +.T.
For class 2 of 41 figure " 2
Forclass 1 of 41fi+ure " 2
"
I
H+I+
i+
fi;~;m+:;;+;++;21
I-tt-I,t -I.I I-H.<-t 4-f I I.I F o r c l m [ of 41f i g u r e ' 3
Forclsm 3 o f • film+re'2 "
+ ,' ",+++
++++, ' +,+" '+,
,++ ,++, "+", +, +,+'+,+ +,+,+",+, ,+,
im++mTm+ +il=++. I . i i~ i]~i+__.+]~i fi+~E2_.._++:i~i+-i +++++tt+t+++++++l p+++1++++ t++i;+tt+titt++ti+..l
For c l • .
2 o f a figme "3 "
Forclass
3ofafigure'3"
Forclass
4ofafilure'3
"
t++' ++++, ++ , +;+,, ++o, l+l ++++ ,:,+,+,
Forclass
5ofafilwe'3"
lu++;.t+++++. +, l+++;.+l
ii!:,L:ii!iL:ii!iii
ii!ii:+ii!ii~ i:~i~ ;i:;;i;;-';i~+ i.';il
~ii~i:+ :1<:H.-:-+4:+i ¢-H-:-t ~ j
i......, i .......
i-t <.H ..~t <.H ÷ t <-H +,-I Forclam I ofafigure'4
Foe' e l m 2 0 f l
+;-.:+!::i++!+::!!+l
mmm+m++.-I I!~
; +:i ;.7.+;:;i M+;~; ,'4 i~1211:11113:~.12
For c l • l l 2 o f • f i l m s "$ "
t++t;+t++tt:+.tell:+.I
g+~+.++I::++. +. +.:++==+++I [t~t+tl+t++.ti+tt+. l+il$;l'P.l$;;:ll$i
it:~1213:11~'-13 ;~'-
Fo~cbm
For c l u s 2 of 41figure " 6 "
I of 41filure " 6
;++;+t++++. ;;:'I;I;'3:;i'Ti;]++++l;+-I ;M l~-;~ ~3:1i,T.~I:t.?.
l
!+~ii~i i~ ++_'i.+
13:1121 112 .":~i?. i++ii+r +1
For c141u 3 of • figure " $ "
Iti~tt~tl+i~It+t~l l++++++.+~]++h-+i+
li~!..+!~!.+!i+i.:+l IL+.+++-t ~++-++:~++-I
For clam $ of • figure " 6 "
For ale841 4 of 41 figure "6 "
t!+,.-+-+-+m+-+i i ++--++++++++~++l+++++MI-~MI+MI-+-I~+' ++++"m++mT"
I ; .'P; t .":'t l ' ; t +'P;;'P; .':'I
r+z.;;++[:;;;;;:;;+.--I
I.I.(H+H-H<.i<-i.+.+.l
For e l m s '; o f • f'ilum " 4 "
;;:;;.,7-; ;i~ ;.~;~
++++++++He+Heel
Forclm~ l of•figure'+
figure "4 "
+++++.++~.tl+.l.+.t++.l
t+++tlt+++t+l+++t+l ;;:I;I;~I~'F i+
l!++.++1,![i+.+ lii+i?+~ 13:~3~-i i12 ~:11:':
~i:ii 13:11~ t:':i2 ii:;i ;~ii:- i:;i~.
13:~121.115 3:~15
1-t<-H tI.+<-H<-I <-~t e l
For class 2 of a figure " 7 "
Fo~cla, ! ofafillure "7
For clam
3 of• figme "7 "
+++++I++++..+++::++-I!i!i+i++ii+i~i+ +I +++++++++++++!++++
;<.+l-b;l.+/4.Fl-+l-b
I
13:11513:1121~:11~
Forcl~, 1 ofafilur+'8
For class 2 of a figure " 8 "
+~:11 i,+,:i ~13:~i~ ;;:4; ;~-'; t'-;i+;~'-
;;:;; ;;;;7- ~;~-
+,i.;+:;i:,I.T.+il.fi,b.T+.ii.';,;.T.mI=+l For class $ of 41 figure " 8 "
+++::++ .++++++:.-+~++I
[+~+l+~i.-++++I++I I+~+l++'++-+++l
[++i i+.+++i++++ i+~+~++.. FI ....... I+I +++: .;~; ~ i ;i-';;~-i; ,3:12,3,2;~-
F H - H 'F H-t" I -'.4 <-:"~+I Forc141, I o f • f i g u r e ' 9
I- t,(-~.I,t } q-FI ~'14-FI -:,.I
"
F o r c141m 2 of 8 figure " 9 "
.................
I"I ¢'H':'I l'~"I <"I11'11":"i Fo~cl•m 3 o l ' a f i l u r e ' 9 "
I"I l'l't 4' i l":"I "~'l ¢'I I 4"1 Forclu•4 ofafilUre'9"
i+I+++I+l ,,.+.. ;~i ;;+;+i+ ,32,:,, ;: i+
'++"+++"'+
I"l 4"1"rIk I 4-:. 14.1.4-I.t 4- I Forelus $ofafllUm'9
- -
: A cross-detectin
I line to require
......
: A omitted cross-detecting
- -
: The frame line.
for recognize the figure successfully. line.
Fig. 11. Allocation of cross-detecting lines after o p t i m i z a t i o n by the selection m e t h o d .
"
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
342
(a)
(b)
N N N N N N N N N N N i l
IIIIINIININNIllll n l l l U l l l l l n l l l
--~--.,;-.~--~.,...-~i~..i~I--iif..~..l .....i i.. _i..~.. --~- .~--~-.;..
. . i.-~--
', ; ; :
I m l i i n l l m l l l l l l
IlUlIINNIlll l l l l l l B l l l l l l l l l
,
.~--4-.
; :
+-i--!--! ............ !-- -i-- -,,:--..:--
I m l l l l l l l l l l l l l l ....
llllliUlinl U l l l U l l l l
... . . . . . .
';'F~i .......... :
l l l l l l l l l l l U l l l l l l l l
.'__~_.
i:
.i..~..i..i..i..i ........ i-- -i-..i--,.':-
iiiiii
ummmummmmmmml
IIIlUlUlIII
i i i ~;
->~~~il .......i ~ ~-*,,
Fig. 12. Common cross-detectinglines after optimization: (a) by the elimination method; (b) by the selection method.
"l'classl
. . . . "O'~lass2...5,...~l;elats~ . u ClaSSl..Q"
"
1 "6"e~
"
/
~
~
~ \
/
~
"~.1.
class~
.
.+~ "4"class2 \....Y'4"~i,,a
-lassl "5"class3
The seloetlon method - - - ~ - - - T h e el iminatlon method Fig. 13. Comparison between the elimination method and the selection method in the number of cross-detecting lines.
5. Conclusion
We proposed the extracting algorithm of membership function assignment based on example-based learning with optimization of cross-detecting lines. Using this algorithm, a fuzzy neuron can be designed very easily without the knowledge about the feature of the character and optimum membership functions can be assigned. As compared with the elimination method, the advantages of the selection method are that the amount of sensor arrays, the amount of the cross-detecting lines for each fuzzy neuron and the C P U time are reduced. Among various conventional methods of character recognition, a zonde method seems similar to the fuzzy neuron. The zonde method carries out recognition by calculation of crisp value, but the fuzzy neuron does the recognition by calculation of fuzzy grade. So, the recognition by the fuzzy neuron is more flexible than the zonde method.
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
343
References [1] T.L. Dimond, Devices for reading handwritten characters, Proc. the Eastern Computer Conf., Washington, DC (1957) 232 237. [2] T.L. Dimond, Reading handwritten characters, Bell Lab. Record 36 (1958) 34-35. [3] M. Furukawa and T. Yamakawa, The advanced method to optimize cross-detecting lines for a fuzzy neuron, Proc. the 5th lnternat. Fuzzy Systems Association World Congress '93, Seoul (1993) 66-69. [4] S. Huang and K. Chuang, Heuristic approach to handwritten numerical recognition, Pattern Recognition 19 (1986) 15 19. [5] T. Kohonen, An introduction to neural computing, Neural Networks 1 (1989) 3-16. [6] Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard and L.D. Jackel, Back-propagation applied to handwritten zip code recognition, Neural Computing 1 (1989) 541~-551. [7] B.T. Mitchell and A.M. Gillies, A model-based computer vision system for recognizing handwritten ZIP codes, Machine Vision and Application 2 (1989) 231-243. [8] D.E. Rumelhart and J.L. McClelland (Eds.), Parallel Distributed Processing, Vol. 1 (MIT Press, Cambridge, MA, 1986). I9] J.D. Tubbs, A note on binary template matching, Pattern Recognition 22 (1989) 359-365. [10] T. Yamakawa, Pattern recognition hardware system employing a fuzzy neuron, in: Proc. lnternat. Conf. on Fuzzy Logic and Neural Networks, Iizuka, Fukuoka, Japan (1990) 30-38. [11] T. Yamakawa and M. Furukawa, A design algorithm of membership functions for a fuzzy neuron using example-based learning, Proc. IEEE Internat. Conf. on Fuzzy Systems 1992, San-Diego, CA (1992) 75-82. [12] T. Yamakawa and S. Tomoda, A fuzzy neuron and its application to pattern recognition, Proc. the Third IFSA Congress, Seattle, Washington, August 6-11 (1989) 943-948.
sets and systems ELSEVIER
Fuzzy Sets and Systems 71 (1995) 329-343
The design algorithms of membership functions for a fuzzy neuron M a s u o F u r u k a w a a'*, Takeshi Y a m a k a w a b =Department of Electrical Engineering, Nagano National College of Technology, Nagano, Japan bComputer Science & Control Engineering, Kyushu Institute of Technology, Fukuoka, Japan
Abstract This paper describes two design algorithms of membership functions for a fuzzy neuron using example-based learning with optimization of allocation of cross-detecting lines. One algorithm is called the elimination method which eliminates the inefficient cross-detecting lines and the other is called the selection method which picks up the efficient cross-detecting lines. The features of these algorithms are that no knowledge of experts is required to design, and the least cross-detecting lines are arranged. We verified these algorithms by recognition of hand-written characters. In comparison with the elimination method, the selection method can reduce the number of cross-detecting lines common to all figures (i.e. the number of the sensor arrays) and the CPU time for design of the membership functions of a fuzzy neuron. Keywords: Fuzzy neuron; Pattern recognition; Design of membership function
1. Introduction The artificial neural networks I-5, 8] employing conventional neuron model or analog threshold element are not designable, although each neuron model is very simple. To cope with this problem, a f u z z y neuron has been proposed [12-1 and applied to a high speed recognition hardware system of hand-written characters 1-10], in which the weighting factors and summation of the mathematical model of neuron are modified by membership functions and M I N operation, respectively and a threshold level of neuron is not assigned. Its features are easy to design and robust against the noises and defects in the pattern. The fuzzy neuron has been implemented in a silicon chip called a f u z z y neuron chip. The fuzzy neuron chip is of very high speed and compact in addition to those features. On the other hand, various methods of character recognition have been developed 1-1,2, 4, 6, 7, 9]. Though most of those methods are too complicated to implement in hardware, the fuzzy neuron is an easy one. Among those methods, the zonde method [1, 2] seems to be similar to recognition by the fuzzy neuron. The advanced distinctions of the fuzzy neuron from the zonde method are no regulation for writing characters and more flexibility by membership functions.
* Corresponding author. 0165-0114/95/$09.50 © 1995 - Elsevier Science B.V. All rights reserved SSDI 0165-01 14(94)00278-9
330
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
Though a membership function makes the fuzzy neuron flexible and designable, know-hows of human experts are necessary for this design. In some applications, we cannot get the know-hows from a human expert or there is no human expert to do so. In those cases, a design algorithm without know-hows of human experts is needed. We proposed two design algorithms of membership functions for the fuzzy neuron using example-based learning with optimization of cross-detecting lines I-3, 11]. These algorithms facilitate system design without the know-how of experts. One of these algorithms is called the elimination method which eliminates the inefficient cross-detecting lines, and the other is called the selection method which picks up the efficient cross-detecting lines. Both algorithms are divided into three parts which are classification of pattern samples, assignment of membership functions and optimization of allocation of cross-detecting lines. Procedures of these three parts are summarized as follows. First, pattern samples of each categories for example-based learning are classified to adequate classes by their outlines. Second, a fuzzy neuron is assigned to each classes and membership functions for each fuzzy neuron are assigned by example-based learning from the cross-points which are provided by classified pattern samples. Thirdly, the optimization of the number and location of crossdetecting lines is achieved. While the procedures of the classification and the assignment are the same in both of those methods, the optimization procedure is distinct from each. The elimination method gives optimum cross-detecting lines by omitting inefficient ones. On the other hand, the selection method gives optimum cross-detecting lines by selecting out efficient ones. In this paper, both of these methods are discussed, implemented by software of EWS, verified by recognition of hand-written numerical characters and compared with each other concerning assigned membership function and CPU time.
2. A fuzzy neuron and its application for pattern recognition of hand-written characters 2.1. A f u z z y neuron
Fig. 1 shows a schematic model of a fuzzy neuron, in which X1,X2, ... ,Xi, ... ,X, are vectors of crisp sets and/or fuzzy sets to inputs, wl, w2. . . . . wi, ..., w, are membership functions of the synaptic weight and Y is an output of the fuzzy neuron ranging from 0 through 1. Excitatory and inhibitory connections give the grade of soft matching/~ between the input Xi and the synaptic weight wi. The grade #i is fed to a MIN block of the fuzzy neuron model. Excitatory connections are represented by MIN, and inhibitory connections are represented by fuzzy logic complements followed by MIN. The output of the fuzzy neuron is a minimum
W,
, y
Fig. 1. A schematic model of a fuzzy neuron.
M Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
331
grade of connections and gives the grade of soft matching between all of inputs and all of membership functions. 2.2. Pattern recognition of hand-written characters by a fuzzy neuron A fuzzy neuron is able to recognize one character by itself. Pattern recognition of hand-written characters by a fuzzy neuron has been discussed [10], in which cross-detecting lines are employed to get features of characters as shown in Fig. 2. There are three regions on the cross-detecting line, a crossing region, a forbidden region and a don't care region. The crossing region is the portion of the cross-detecting line where the character should cross. The forbidden region is the portion of the cross-detecting line where the character should not cross. The don't care region is the portion of the cross-detecting line where the character will uncertainly cross. One or more cross-detecting lines are assigned in the frame. The cross-detecting lines of all characters, which are put into the same input frame, are referred to common cross-detecting lines. The boundaries of these three regions are fuzzy, so that they should be defined by membership functions for each cross-detecting line as shown in Fig. 3(a). If there are two or more crossing regions on one cross-detecting line, the membership function assigned on that cross-detecting line should be divided into
c.o,o . oo. 7
._SOT
A t ~ "-..-'-..~.i ............................ --
,' .-"
B
. oo.
f . .............. ......,
".. ............ iy"
q
%..
pl
i
_z
(a) 1I 0 #2
$
W U) O n.
CROSSING REGION
o
FORBIDDEN R "GION
" i
~
c!i 'o'oo "'ro" /i )J
#3
C
/
(c) 11 0 III
T il
!a
I I I I I I I I I I I I
[b
lle I I I I I I
~c
liB I ~
CROSSPOI CROSSING REGIONJ CROSS-DETECTINGLINE
1
Fig. 2. Cross-detecting line in the frame.
SENSORARRAYIN\ CROSS-DETECTINGLINE ~ Fig. 3. Assignment of the membership function of a crossing region.
332
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
FORBIDDENREGIO~'-~~ A R E
BREGION'~O~BIDDEN REGION "7
/~4
i
: i
0 F5
!d
'
!b
w z II¢3 '"
ie
i7
CROSSPOINTS SENSORARRAYIN CROSS-DETECTINGLINE
Fig. 4. Assignment of the membership function of a forbidden region.
two or more membership functions as shown in Fig. 3(b) and (c) in order that a cross-point of the don't care region does not affect the results. These membership functions are used for excitatory connections. Fig. 4 shows a membership function of two forbidden regions. The membership function of forbidden regions are used for inhibitory connections. The don't care region is not used for recognition of characters, because it has no information. Membership functions for crossing regions and forbidden regions can be assigned on a cross-detecting line. The appropriate number of cross-detecting lines are located horizontally and vertically in the frame as common cross-detecting lines. Cross-points detected by sensor arrays arranged on common cross-detecting lines are applied to input buses of a fuzzy neuron. An output of each fuzzy neuron represents the grade of the possibility of the character.
3. Automatic design algorithm of membership functions using example-based learning with optimization of cross-detecting lines Fig. 5 shows design flow of membership functions using example-based learning with optimization of cross-detecting lines. In example-based learning, when the variance of pattern samples is big, the feature of the patterns defuses and is very difficult to squeeze out. To cope with this problem, each category of pattern samples should be classified to adequate classes. And a fuzzy neuron is assigned to each classes. In other words, the number of fuzzy neurons needed to achieve a reasonable pattern recognition is equal to that of whole classes. Thus membership functions for a fuzzy neuron should be assigned from each class.
M. Furukawa, T. YamakawaI Fuzzy Sets and Systems 71 (1995) 329-343 CATEGORY "0"
CATEGORY"1"
333
CATEGORY "9"
i 0 i 3, iiiiiii!ii iiiiiiiilililili.......... PATTERN SAMPLES
~ ..o ~'"'"'I
~--.r
.......
• .. INITIALi..~
PAI"rERN SAMPLES
PATTERN SAMPLES
.......l .......
~
I
I ~'~'~'~'°"
I
~................ ,"
.......1........ -I
........
~ --- ~
. . . 1
i'"i ~t~'~s ~ ~,J,cno,s ~"'i ~,er~
i ..........
~ ~J,c.~o,s i"" i FU,e~....S....
OPTIMIZATIONOFALLOCATIONOF CROSS-DETECTINGLINES
OPTIMUM MEMBERSHIP FUNCTIONS (Cluster Z1)
MEMBERSHIP FUNCTIONS (ClusterZ2)
• • • • • • • • •
MEMBERSHIP FUNCTIONS (Cluster Nn 9)
Fig. 5. Design flow of membership functionsfor a fuzzy neuron. Pattern samples of each class give membership functions in all the cross-detecting lines on the each frame. Membership functions are assigned from cross-points of a cross-detecting line and pattern sample. A frame carries many cross-detecting lines and membership functions are assigned on all cross-detecting lines. It means that the fuzzy neuron has the same amount of synaptic membership functions or more. However, all of the membership functions are not necessary for a reasonable pattern recognition depending upon the complexity of the pattern and the number of the categories to be distinguished. Thus, thirdly, the optimization of the number and location of cross-detecting lines is achieved. 3.1. Assignment of membership functions using example-based learning 3.1.1. Classification of pattern samples in a category Pattern samples can be classified by a human at a glance or by the computer which executes the following algorithm.
334
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329 343
The cross-detecting lines are assigned in a frame to classify pattern samples. In this paper, 64 horizontal cross-detecting lines and 64 vertical cross-detecting lines are assigned to classify pattern samples. The points of a character on the cross-detecting line, which are crisp, are fuzzified into a trapezoidal membership function to compare rough outlines of a character. In order to define the shape of this membership function, let L represent a width or a height of the frame. represents a fuzzification coefficient which takes the value ranging from 0 to 1.0. t represents a width of a character crossing to the detecting line. Then the support of the membership function will be ~L + t, as shown in Fig. 6. After all pattern samples are fuzzified, the pattern samples in a category are classified into some classes by a similarity measure Sij between the ith and the jth fuzzified pattern samples. A similarity measure Sij is defined as [ ~e'= I ]Ae'l'i A ]Ae,l,j 1 ~ e = 1 [2eli A [~elj 71SiJ -- g -4-~ s 1 / = 1 ~u= 1 12eli v lieu l'= ~ eU''= 1 12e'l'i V ~te,l, j J
,
(1)
where lleti represents the eth element of a sampled membership function on the/th horizontal cross-detecting line in the ith pattern sample, U represents the number of elements in the horizontal universe of discourse, or the number of horizontal cross-detecting lines in the class. The ' represents vertical ones. Step 1: The first pattern sample is regarded as a reference and Slj is calculated for all j's except this reference. Step 2: The first pattern sample and the others which satisfy the condition Slj > [Sd]k,
should be included in one class, where [Sa] k represents the threshold similarity measure for discrimination in the kth class. Usually [Sd]k'S for all classes are equal to each other, that is, [Sd]k = Sa. Step 3: The residual pattern samples are renumbered. Step 4: Repeat Steps 1-3 until the classification is completed.
L
0
/\ i o,' Li iaLi i "2-.i~ t.i~ 2~.i
Fig. 6. Membershipfunction to fuzzifya pattern for clustering.
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
335
A human can classify by comparison the features of the patterns at a glance. The features which are employed by human to classify, are line width of the pattern, curve rate of the line, tilt of the pattern and size of the pattern for example. The result of the proposed classification is close to that of what the human classifies, if • and Sd are suitable values. The similarity between rough outlines of patterns in the same class is roughly proportional to Sd and reciprocal ~ values. The values of ~ and Sd are assigned under the following conditions: (a) The number of classes is close to that of what a human classifies. (b) The rough outline of patterns are similar to each other in the same class. (c) ct is assigned for each membership function not to overlap with neighbor one on the same cross-detecting lines.
3.1.2. Assignment of membership functions of a fuzzy neuron Classified patterns will provide us with cross-points on horizontal and vertical cross-detecting lines corresponding to the class of pattern. The membership functions for a crossing region and a forbidden region can be assigned from these cross-points. Fig. 7 (a) shows, for example, only one (vertical) cross-detecting line with cross-points produced by three samples of hand-written character "3". The lines where membership functions are obtained by this assignment procedure are called initial cross-detecting lines, because they should be optimized. First, in order to assign the membership function for a crossing region, these cross-points are clustered by a simple method with the cluster radius rd as follows: Step 1: The first data (positions of cross-points, al,a2,a3,a4) of character in a class is located on a cross-detecting line. It is examined that each point is located inside of clusters, the center of which is al, a2, a3 or a4. If each cross-point is not included in each cluster (independent from each other), all of these cross-points can be centers of clusters. If some cross-points are included in other clusters, these cross-points should be regarded to be in one cluster and a virtual center of this cluster is a mean position of these cross-points. In case of Fig. 7, for instance, a2, a3 are in the same cluster, the center of which is a midpoint of a2 and a3 on a cross-detecting line. Step 2: It is examined that the next data (positions of cross-points, bl, b2, b3) of character in the class are included in the clusters of radius rd defined in Step 1. In the similar manner, all the data are examined to be included in the clusters. Step 3: The clusters should be classified to a crossing region or, a don't care region. If all the pattern samples give one or more cross-points to the cluster, it is a crossing region. Otherwise, it is a don't care region. In case of Fig. 7, cross-points are classified to three clusters (regions). The first cluster (al,bl, cl) and the third cluster (a4, b3, c2) include all the pattern sample, so that these region are crossing region. The second cluster (a2, a3, b2) does not include pattern sample "c", so that this cluster (region) is a don't care region. Step 4: A membership function of crossing region is assigned as follows. The outer elements (al and bl, and b3 and c2 in Fig. 7) of cross-points in the crossing region can be the shoulders and give the spreads outside by ~L/2 as shown in Fig. 7(b), where t represents the distance between outer elements of cross-points, L represents the length or width of the frame, • represents fuzzification coefficient. Thus the support of the membership function is ~L + t. There is no need to assign the membership functions for don't care regions, because they are not used for recognition. The forbidden regions can be assigned only if the distance between shoulders of neighboring crossing regions are greater than 2fl + ~L as shown in Fig. 8. In this figure fl is a constant representing a distance between the spread of the forbidden region and the shoulder of neighboring crossing region. The spreads of the forbidden region are the points outer the shoulders by ~L/2 as those of the crossing regions.
336
M. Furukawa, T. Yamakawa/ Fuzzy Sets and Systems 71 (1995) 329 343
(a)
- - CROSS-DETECTING LINE
a R .................................
. . . . . . . . . .
bl cl a2 a3 b2 b3 a4 c2-
ILL;
1I.---.~.
, (b) DON'T CARE REGION CROSSING R E G I O N ~ CROSSING REGION al blcl a2 a3b2 b3 a4c2
~--T
-
--
i
T--T
o
\ iaLi
f~ a"L+t ,:'~ Fig. 7. Assignmentof membershipfunctionfor crossing regions. (a) Cross-points between character "3" and a cross-detecting line. (b) Membershipfunction for crossing regions.
-
aL 2
iaL 2 i
i
GL~ 21
,K 9 Fig. 8. Assignment of a membership function for a forbidden region (solid line).
3.2. Optimum allocation of cross-detecting lines A frame carries many cross-detecting lines (typically 64 x 2 = 128 lines). It means that the fuzzy neuron has the same amount of synaptic membership functions which are obtained by the procedure described in Section 3.1.2. However, all the membership functions are not necessary for a reasonable pattern recognition depending upon the complexity of the pattern and the number of the categories to be distinguished. Thus it is possible to reduce the cross-detecting lines, i.e. membership functions for the efficient recognition. The procedure for this optimization of allocation of cross-detecting lines is discussed in this section. In this paper, the number of horizontal cross-detecting line and vertical cross-detecting line are 15 and 15 for saving C P U time, respectively. (However, in Section 3.1.1, 64 cross-detecting lines are assigned for each direction, to classify pattern samples with similarity measure.) These cross-detecting lines are assigned in a frame uniformly, as shown in Fig. 9. This optimization is carried out by different procedure for the elimination method and the selection method.
3.2.1. Optimization by the elimination method To explain optimization by the elimination method, following criterion for eliminating of cross-detecting line is presented. Criterion for eliminating cross-detecting line (CEL): The following notation is assigned. X(k): a pattern or a character X of the kth class which should be recognized; xdk): the ith pattern sample in the kth class X(k), i.e. xi(k) ~ X(k); FNX(k): a fuzzy neuron designed compatible to a class of character X(k); OUT(X(k); xi(k)): an output of F N X ( k ) when xi(k) is input;
M. Furukawa, "1~ Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343 VIV273
•
• 77
337
V15
H1 H2 H3
H7
H15
Fig. 9. A s s i g n m e n t of cross-detecting line.
Y(/):
a pattern or a character Y of the/th class which is out of the category including a class X(k); y j(1): the jth pattern sample in a class Y(I), i.e. yj(l)E Y(I); OUT(X(k); yj(/)): an output of FNX(k) when yj(l) is input; d: a minimum distance between OUT(X(k); xi(k)) and OUT(X(k); yfl)) (typically 0.9). The criterion: If the cross-detecting line under the test is omitted and A OUT(X(k); x,(k)) - V V V OUT(X(k); yj(/)) >~ d, i
r
l
(2)
j
then the cross-detecting line should be omitted. Otherwise, the cross-detecting line should be held. Checking of cross-detecting lines in case of Fig. 9 is achieved in the following sequence using the CEL. If the number of cross-detecting line is changed, the order of checking of cross-detecting lines should be determined in the similar manner. Step 1: The horizontal cross-detecting line of the center of the frame is examined by the CEL. Step 2: The vertical cross-detecting line of the center of the frame is examined by the CEL. Step 3: The horizontal cross-detecting line of the center of the upper half of the frame is examined by the CEL. Step 4: The vertical cross-detecting line of the center of the left half of the frame is examined by the CEL. Step 5: The horizontal cross-detecting line of the center of the lower half of the frame is examined by the CEL. Step 6: The vertical cross-detecting line of the center of the right half of the frame is examined by the CEL. Step 7: The horizontal cross-detecting line of the center of the top quarter of the frame is examined by the CEL. Step 8: The vertical cross-detecting line of the center of the most left quarter of the frame is examined by the CEL.
Step 29:" The horizontal cross-detecting line of the center of the bottom eighth of the frame is examined by the CEL.
Step 30: The vertical cross-detecting line of the center of the most right eighth of the frame is examined by the CEL.
338
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
3.2.2. Optimization by the selection method The selection method is achieved by two procedures. First, the procedure for optimization of common cross-detecting lines is carried out. In this procedure, common cross-detecting lines are selected out from initial cross-detecting lines as efficient cross-detecting lines for recognition. Second, the procedure to obtain optimum allocation of cross-detecting lines of each fuzzy neuron is carried out. In this procedure, efficient cross-detecting lines for each fuzzy neuron are selected out from common cross-detecting lines. To explain these procedures, the following notation is assigned. CDL(mi.it): the minitth initial cross-detecting line; CDL(m~om): the mcomth common cross-detecting line; X(k): a pattern or a character X of the kth class which should be recognized; xi(k): the ith pattern sample in the kth class X(k), i.e. xi(k)e X(k); CDL(X (k), mopt): the moptth optimum cross-detecting line for the kth class X; FN X(k) CDL(minit): a fuzzy neuron using only one cross-detecting line CDL(mi.it) of a class of character X(k); OUT(X(k) CDL(mi,it); xi(k) ): an output of FN X(k) CDL(minit) when xi(k) is applied to the input; Y(/): a pattern or a character Y of the /th class which is out of the category including a class X(k); thejth pattern sample in a class Y(l), i.e. yj(l)e Y(I); Yfl): OUT(X(k)CDL(m~.it); yj(l)): an output of FNX(k) CDL(miait) when yj(l) is applied to the input; FN X(k): a fuzzy neuron designed compatible to a class of character X(k); d': a minimum difference between OUT(X(k)CDL(mi,it); xi(k)) and OUT(X(k)CDL(mi.it); yj(l)) (typically 0.9); NUM(X (k)CDL(mi,it); Y(/)): the number of the classes Y(l) including pattern samples which are not accepted as a character X(k) by FN X(k)CDL(mi.it), i.e. the number of the classes Y(l) which satisfy
A OUT(X(k)CDL(m~°~t); x,(k)) - V OUT(X (k)CDL(m~.it); y fl)) >~d' ; i
(3)
j
NUM(X(k)CDL(m¢om); Y(I)): the number of the classes Y(l) including pattern samples which are not accepted as a character X(k) by FN X(k)CDL(m¢o,n). i.e. the number of the classes Y(l) which satisfy AOUT(X(k)CDL(m¢o~); x,(k)) - VOUT(X(k)CDL(m¢om); y.i(1)) >t d'. i
(4)
j
The procedure for optimization of common cross-detecting lines is as follows: Step 1: Calculate the sum of NUM(X(k)CDL(ml.it); Y(1)) for all mi,it's by
N .....
=
~ ~ NUM(X(k)CDL(m~.~,); Y(l)).
(5)
X k
Step 2: Pick up the cross-detecting line CDL(minit) which has the maximum value of N ..... in initial cross-detecting lines as one of the CDL(mcom). Step 3: Discard the class Y(l), which satisfies the Eq. (4), from Y(l). Step 4: Repeat Step 1-Step 3 until Y(l) becomes empty. The procedure to obtain optimum allocation of cross-detecting lines of FN X(k) is described by the following steps: Step 1: Calculate the sum of NUM(X(k)CDL(mcom); Y(l)) for all mcom'Sby N .... = NUM(X(k)CDL(m~om); Y(l)).
(6)
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
339
Step 2: Pick up the cross-detecting line CDL(mcom) which has the maximum value of Nm°o. in all moore'S as one of the CDL(X(k), mopt). Step 3: Discard the class Y(1), which satisfies from Y(l), A OUT(X (k)CDL(X (k), mopt), xi(k) ) - V OUT(X (k)C DL(X (k), mopt), yj(l) ) >t d'. i
(7)
j
Step 4: Repeat Step 1-Step 3 until Y(l) becomes empty.
4. Experimental results
4.1. Optimum allocation of cross-detecting lines and assignment of membership functions The selection method and the elimination method are implemented by an EWS. The algorithm of pattern recognition is implemented by a PC. Patterns for recognition are ten figures which are from "0" through "9". The programming language is the C language. The data of patterns and membership functions can be transferred between EWS and PC each other. To verify the algorithms, membership functions of ten figures which have been assigned by example-based learning were optimized by both methods. Twenty pattern samples for each figure have been used and the pattern samples have been classified into 35 classes by their outline. Where 20 pattern samples were employed for one figure, sizes of the frame to input a pattern are 64 x 64 dots and ~t, fl, ra and d are 0.1875, 0, 10, and 0.9, respectively. In order to estimate the recognition rate for the patterns which are not employed to learn membership functions, we provided five pattern samples for each figure (i.e. 50 pattern samples for ten figures). Fig. 9 shows locations of horizontal and vertical cross-detecting lines which are assigned initially to optimize cross-detecting lines. There are 15 cross-detecting lines for horizontal and vertical before optimization. Then, the recognition of hand-written characters was carried out using the membership functions after optimization. Figs. 10 and 11 show locations of cross-detecting line for all classes of ten figures after optimization by the elimination method and the selection method, respectively. A few cross-detecting lines for horizontal and vertical are required to recognize the figure successfully. Obtained cross-detecting lines for each categories, shown in Fig. 10 for the elimination method and Fig. 11 for the selection method should be merged to the common cross-detecting lines as shown in Fig. 12(a) and (b). And sensor arrays should be located as shown in Fig. 12(a) or (b).
4.2. Comparison between two methods Fig. 13 shows the number of the cross-detecting lines of each fuzzy neuron after both optimizations. The selection method decreased the number of the cross-detecting lines of each fuzzy neuron in thirteen classes from it of the elimination method. But, in three classes, the selection method increased it. The number of the sensor arrays by the selection method was 38% less than the elimination method. The processing time for the design by the selection method was 1 h and 27 min and that of the elimination method was 3 h and 47 min. Thus, the processing time of the selection method was 62% shorter. Both of these methods can recognize the patterns which are employed to learn membership functions, completely. In the recognition experiment for the patterns which are not employed to learn membership functions, the recognition rate is 82% for the selection method and 86% for the elimination method.
340
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
Ii i il i. ii::ii. l L~:LL+ I-2L+L~:.r!±
I
;~:gi-"/i ~;Ti~:;i~.
;r-;;~;r-~;T ;:~i~ Forclass
~-~;÷; -~÷--~i" I
I ofafilu~e'0
"
r o t class 2 of • figure "0 "
p,il ii,ii ii,I 14-~. I # 14~bl-,b ,l-ll 4-
i .......... | .... : For elm I ofa filiure " I "
|~:~i$i :~i; l,":il; I;:IIT; il;~:;;~ For c l a m 2 o t ~ fisure " l "
,~-~,~-, HTH-.-T-
;iTiTl ;i~-;Ti~ I~:~I.T.I ~t~ ~:~t~Fo~clms
3 ofafiEure'l
"
Forclass 4 o f a f i s u r e " l
"
li:ll ll:ll?.ll. I~-
HIIIIIIJI II-~.I
For class [ oi" a fl~lure " 2 "
II.$1-1-II. |.1.
For class 2 o / a fil~Ure " 2 "
I..............[i; i~-'ii$;;T;~iT
L:fi ~--[~ i:':iii ff~_-:iiii: ~:~Zr I ol'a filu~e'3
"
i ........
r
ii:iiii~
ii:ii~-ii:i ,7 ~ i-Tii:il$11:i $ ~ ii
ii:liT;TI
F o r class 2 o f a fillure " 3 "
Forcl~s
fii::ii i ii li i:=:I P''T
i;i:.;17. Tii~i-T-
11:iIIi~, $ii.~ii 11:;1-7.;~ -?liT;;
IIT;7-IT; -T- ;T
I. I .I-I I + 14-I. I ~.I I-H~.-I Forclan
~:.:.::.~.: .-'~6.::-::L=
i ......
~o, ~.l~:.3..o.r:n.pff-~i/:ilZ-h":l ,~ ~ i-~
i :':l i -?.i ,~:.~~.T.i 1: i;
;T;i ig~;T TgiT
i;:~;;;]ii;;;;;z. I 14.11,¢.I -I 1-1-1-1-:-I-1-
~ofafillU~']"
Forcl~s
4ofafilu~'3"
For clus
S ofafilum
"3
fii::ii ilii i i. l
14.~.14-1.l.ll.~- I l . l l 4 -
I l.ll'l'l
11~'14"II+
liiii,ilii,ii, liT;T; ;iT;Ni,7
1.14:H..~ I ¢::-I-:- II:F i + I For c l a ~ I o i ,
Illume " 4 "
For class 2 o f
"4 "
s filiute
i::::::l::::::III
For class 3 o f I fi~ure " 4 "
Iiiiii it!i!! !! l¢.ll4-I
;TIII;TI;~.I~:;;~ i~:ii~.;l:li '~i~:~i~ l:orclm
14-~.I-:. 14-} l.l- I¢.~. I -:.
:i;:i;.~i;:~;T;i:;i$
I o f a failure " 5 "
For class 2 o f a f i l u r e " $ "
ll~.ll-ll-:-
hi ii ib:ii
i ...... | ........ For c l ~ a ~ o f a f i ~ v ~
i
"~ "
iiNiiii iiiI i
,.~:,.,.2 2 ,.2 ,.:,.,..+I For class I o t a f i l u r e " 6 "
For ¢l~s 2 ola fillwe "6 "
I
fiitiii i. l itl
i .......... i .... : Forcl~s ~ of a ilium'6
"
................. For e l m 4 o f o r q l u ~ ' 6
"
Iii li:~iiiil~ii~i~l I~ 14-I¢-~1¢-; 4-~1-;
i~ i$;Tii-Tii:llT Q,.I
For clm
. . . . . . . . . . . .
lli
il.'Ii-T; lIT ~iT
;;T;T; .......
i
1 o f a failure " 7 "
i,'.:~$i ;i$ ~i$
;;;;:;i.:: ........
For class 2 of a filure = 7 "
Fo~clul
] o~a f i l u r e " 7 "
J . : l - ~ I J - : ; 4 ; J - : 1 4 : J-
I-; ~-~1":-~I t ¢ 441-;; ~-I
IT;iT; l T ITIT
~ '~}4
~'}-H~,
-
I'I 4":' I ~" I ~l'll"el 1-~"I ":"
f!!!!!~!!~1.-'1~i,z For©Ira
I ota fillur~'ll
l "
~ ,He t4-t ;I.~
Fo~ class 2 o [ a fiSut¢ " 8 "
iTI bT.ii:;;..7;giir. i i i-'i; ~-;i:i i T; TI;r.. il:;;T;I:II~;TI;~ For c l ~ l l 3 a i m l i l u ~
"S "
i ~.%T.i |$i~:~|$ ; ~ i.T-; i.T-I;~;$ 14-$1 I '1.;.14-;,14.
, ......
F o r c l a s s I o f a fisur= " 9 "
For c l a .
.
....
.,..,
2 o f a fisure " 9 "
I" II'I" I "I' I l ' } l ' l " II'F I ~.'I F o r c l m s 3 oJ'a fillUre " 9 "
I" I'I'I" I I" I f l ' l + | 4 " l ' I'P Fol'chuls 4 o f a r l l u r e " 9 "
r..... l i ....... F o r claaa S o~r a fiSule " g "
: A cross-detecting line to require for recognize the fisure successfully. ...... : A omitted cross-detecting fine. --
Fig. 10. A l l o c a t i o n o f c r o s s - d e t e c t i n g lines after o p t i m i z a t i o n
: The frame line.
by the elimination method.
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
++m+l~+++,,m;:,=l i+m+.It.-+m+++++l
I+m~+.-I++I .+, ._+++++++I t!:~!±l.~:t I±.~,:t!± I-I'4-Fl-:- I 1.:-I'4- l 4-I" ":+3:11~. ,+,:i 2~i:~t2
r,+ll,,..:.+:,,
f,~+;.+H++.+I
Forclam
341
I of 41filure "O
For class 2 of 41 figure "0 "
Imtl;m+. p+-.H+tl:.-++++_+ ;;:;; ;;:.q-,7l+++tml ;:;i-'7. E+mI+i+i~im~l lim.;inlmml H+m. +l+;:;;Z.++++.:l ;;:;; if+iT13:11 i,+,:11+...,+.:i~?. 11:11 ~ i .:.:i:112 ~i~1~.1~12,3,12 I 1,3112,H,+I +;;+;+;~-ii~-;<-~i~-i H,++I +,+,,++, Iti.-+;It+It,mt:+l Iiitiii~i:.+:ii+i+l I~mlmm+,l i,+,:11 ;3:+1+i< i+ ;~;:r ;3:I;+;;:+;Z++++1111:++. :+,<.+-I+11
+++L+i+i;+i~i++:l
;~:;; ;~;-~;;-';;;
,<-.+,<-~ + . 1 1 1
;;.';;~,i ;i;;< 1;
Porc~.
For ¢ l • m 2 o/'41 figure " l "
L of 41fillUme'l
Forclan
$ofafigure'l
"
Forclms 4 o f a f i g u r e ' l
"
+:,.+,+++":.:q,~,:+, I++. +l++:++++++++l ~"+i!.?!.'+.!? !":':".? I+++ll!++l++l+tl,.-+.l ~;i:;;t;i:i;z.;~iz.I I++::++l+++}.-.+l+:+++l++++_+++++t+,++~++l +++++;:+-ml+l++++l+m;+.ml=.+tm+.l+;.:+:2Z'.-.IL+I,+.I,2,~.+,++ 13:11 13212 i:13.'+ill,+.' +++-.+l -+.+ll++l-.++l+<.+++++-++++l +-.ll+.l
~+.;:+.;l;,111.+,.;.+:i++:+I
;; i~-;;:;iZ i;-~i+-
1++ : +":+.~:++":~.':: +.T.
For class 2 of 41 figure " 2
Forclass 1 of 41fi+ure " 2
"
I
H+I+
i+
fi;~;m+:;;+;++;21
I-tt-I,t -I.I I-H.<-t 4-f I I.I F o r c l m [ of 41f i g u r e ' 3
Forclsm 3 o f • film+re'2 "
+ ,' ",+++
++++, ' +,+" '+,
,++ ,++, "+", +, +,+'+,+ +,+,+",+, ,+,
im++mTm+ +il=++. I . i i~ i]~i+__.+]~i fi+~E2_.._++:i~i+-i +++++tt+t+++++++l p+++1++++ t++i;+tt+titt++ti+..l
For c l • .
2 o f a figme "3 "
Forclass
3ofafigure'3"
Forclass
4ofafilure'3
"
t++' ++++, ++ , +;+,, ++o, l+l ++++ ,:,+,+,
Forclass
5ofafilwe'3"
lu++;.t+++++. +, l+++;.+l
ii!:,L:ii!iL:ii!iii
ii!ii:+ii!ii~ i:~i~ ;i:;;i;;-';i~+ i.';il
~ii~i:+ :1<:H.-:-+4:+i ¢-H-:-t ~ j
i......, i .......
i-t <.H ..~t <.H ÷ t <-H +,-I Forclam I ofafigure'4
Foe' e l m 2 0 f l
+;-.:+!::i++!+::!!+l
mmm+m++.-I I!~
; +:i ;.7.+;:;i M+;~; ,'4 i~1211:11113:~.12
For c l • l l 2 o f • f i l m s "$ "
t++t;+t++tt:+.tell:+.I
g+~+.++I::++. +. +.:++==+++I [t~t+tl+t++.ti+tt+. l+il$;l'P.l$;;:ll$i
it:~1213:11~'-13 ;~'-
Fo~cbm
For c l u s 2 of 41figure " 6 "
I of 41filure " 6
;++;+t++++. ;;:'I;I;'3:;i'Ti;]++++l;+-I ;M l~-;~ ~3:1i,T.~I:t.?.
l
!+~ii~i i~ ++_'i.+
13:1121 112 .":~i?. i++ii+r +1
For c141u 3 of • figure " $ "
Iti~tt~tl+i~It+t~l l++++++.+~]++h-+i+
li~!..+!~!.+!i+i.:+l IL+.+++-t ~++-++:~++-I
For clam $ of • figure " 6 "
For ale841 4 of 41 figure "6 "
t!+,.-+-+-+m+-+i i ++--++++++++~++l+++++MI-~MI+MI-+-I~+' ++++"m++mT"
I ; .'P; t .":'t l ' ; t +'P;;'P; .':'I
r+z.;;++[:;;;;;:;;+.--I
I.I.(H+H-H<.i<-i.+.+.l
For e l m s '; o f • f'ilum " 4 "
;;:;;.,7-; ;i~ ;.~;~
++++++++He+Heel
Forclm~ l of•figure'+
figure "4 "
+++++.++~.tl+.l.+.t++.l
t+++tlt+++t+l+++t+l ;;:I;I;~I~'F i+
l!++.++1,![i+.+ lii+i?+~ 13:~3~-i i12 ~:11:':
~i:ii 13:11~ t:':i2 ii:;i ;~ii:- i:;i~.
13:~121.115 3:~15
1-t<-H tI.+<-H<-I <-~t e l
For class 2 of a figure " 7 "
Fo~cla, ! ofafillure "7
For clam
3 of• figme "7 "
+++++I++++..+++::++-I!i!i+i++ii+i~i+ +I +++++++++++++!++++
;<.+l-b;l.+/4.Fl-+l-b
I
13:11513:1121~:11~
Forcl~, 1 ofafilur+'8
For class 2 of a figure " 8 "
+~:11 i,+,:i ~13:~i~ ;;:4; ;~-'; t'-;i+;~'-
;;:;; ;;;;7- ~;~-
+,i.;+:;i:,I.T.+il.fi,b.T+.ii.';,;.T.mI=+l For class $ of 41 figure " 8 "
+++::++ .++++++:.-+~++I
[+~+l+~i.-++++I++I I+~+l++'++-+++l
[++i i+.+++i++++ i+~+~++.. FI ....... I+I +++: .;~; ~ i ;i-';;~-i; ,3:12,3,2;~-
F H - H 'F H-t" I -'.4 <-:"~+I Forc141, I o f • f i g u r e ' 9
I- t,(-~.I,t } q-FI ~'14-FI -:,.I
"
F o r c141m 2 of 8 figure " 9 "
.................
I"I ¢'H':'I l'~"I <"I11'11":"i Fo~cl•m 3 o l ' a f i l u r e ' 9 "
I"I l'l't 4' i l":"I "~'l ¢'I I 4"1 Forclu•4 ofafilUre'9"
i+I+++I+l ,,.+.. ;~i ;;+;+i+ ,32,:,, ;: i+
'++"+++"'+
I"l 4"1"rIk I 4-:. 14.1.4-I.t 4- I Forelus $ofafllUm'9
- -
: A cross-detectin
I line to require
......
: A omitted cross-detecting
- -
: The frame line.
for recognize the figure successfully. line.
Fig. 11. Allocation of cross-detecting lines after o p t i m i z a t i o n by the selection m e t h o d .
"
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
342
(a)
(b)
N N N N N N N N N N N i l
IIIIINIININNIllll n l l l U l l l l l n l l l
--~--.,;-.~--~.,...-~i~..i~I--iif..~..l .....i i.. _i..~.. --~- .~--~-.;..
. . i.-~--
', ; ; :
I m l i i n l l m l l l l l l
IlUlIINNIlll l l l l l l B l l l l l l l l l
,
.~--4-.
; :
+-i--!--! ............ !-- -i-- -,,:--..:--
I m l l l l l l l l l l l l l l ....
llllliUlinl U l l l U l l l l
... . . . . . .
';'F~i .......... :
l l l l l l l l l l l U l l l l l l l l
.'__~_.
i:
.i..~..i..i..i..i ........ i-- -i-..i--,.':-
iiiiii
ummmummmmmmml
IIIlUlUlIII
i i i ~;
->~~~il .......i ~ ~-*,,
Fig. 12. Common cross-detectinglines after optimization: (a) by the elimination method; (b) by the selection method.
"l'classl
. . . . "O'~lass2...5,...~l;elats~ . u ClaSSl..Q"
"
1 "6"e~
"
/
~
~
~ \
/
~
"~.1.
class~
.
.+~ "4"class2 \....Y'4"~i,,a
-lassl "5"class3
The seloetlon method - - - ~ - - - T h e el iminatlon method Fig. 13. Comparison between the elimination method and the selection method in the number of cross-detecting lines.
5. Conclusion
We proposed the extracting algorithm of membership function assignment based on example-based learning with optimization of cross-detecting lines. Using this algorithm, a fuzzy neuron can be designed very easily without the knowledge about the feature of the character and optimum membership functions can be assigned. As compared with the elimination method, the advantages of the selection method are that the amount of sensor arrays, the amount of the cross-detecting lines for each fuzzy neuron and the C P U time are reduced. Among various conventional methods of character recognition, a zonde method seems similar to the fuzzy neuron. The zonde method carries out recognition by calculation of crisp value, but the fuzzy neuron does the recognition by calculation of fuzzy grade. So, the recognition by the fuzzy neuron is more flexible than the zonde method.
M. Furukawa, T. Yamakawa / Fuzzy Sets and Systems 71 (1995) 329-343
343
References [1] T.L. Dimond, Devices for reading handwritten characters, Proc. the Eastern Computer Conf., Washington, DC (1957) 232 237. [2] T.L. Dimond, Reading handwritten characters, Bell Lab. Record 36 (1958) 34-35. [3] M. Furukawa and T. Yamakawa, The advanced method to optimize cross-detecting lines for a fuzzy neuron, Proc. the 5th lnternat. Fuzzy Systems Association World Congress '93, Seoul (1993) 66-69. [4] S. Huang and K. Chuang, Heuristic approach to handwritten numerical recognition, Pattern Recognition 19 (1986) 15 19. [5] T. Kohonen, An introduction to neural computing, Neural Networks 1 (1989) 3-16. [6] Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard and L.D. Jackel, Back-propagation applied to handwritten zip code recognition, Neural Computing 1 (1989) 541~-551. [7] B.T. Mitchell and A.M. Gillies, A model-based computer vision system for recognizing handwritten ZIP codes, Machine Vision and Application 2 (1989) 231-243. [8] D.E. Rumelhart and J.L. McClelland (Eds.), Parallel Distributed Processing, Vol. 1 (MIT Press, Cambridge, MA, 1986). I9] J.D. Tubbs, A note on binary template matching, Pattern Recognition 22 (1989) 359-365. [10] T. Yamakawa, Pattern recognition hardware system employing a fuzzy neuron, in: Proc. lnternat. Conf. on Fuzzy Logic and Neural Networks, Iizuka, Fukuoka, Japan (1990) 30-38. [11] T. Yamakawa and M. Furukawa, A design algorithm of membership functions for a fuzzy neuron using example-based learning, Proc. IEEE Internat. Conf. on Fuzzy Systems 1992, San-Diego, CA (1992) 75-82. [12] T. Yamakawa and S. Tomoda, A fuzzy neuron and its application to pattern recognition, Proc. the Third IFSA Congress, Seattle, Washington, August 6-11 (1989) 943-948.