- Email: [email protected]
Inference of fuzzy regular pattern grammar
Pattern Recognition Letters 2 (1983) 27-32 North-Holland
October 1983
Inference of fuzzy regular pattern grammar A.K. MAJUMDAR
and A.K. ROY
Indian Institute o f Technology, Kharagpur, India-721302 Received 7 February 1983 Revised 30 April 1983
Abstract: This paper contains an approach to the inference of fuzzy pattern grammar from a set of sample strings. The grammar is inferred from a finite submatrix of the fuzzy Hankel matrix of the coefficients of formal power series representing the set of sample strings from each pattern class. Key words and phrases: Inference, formal power series, pattern grammar, Hankel matrix, fuzzy grammar.
Introduction
Preliminaries of formal power series
The problem of grammatical inference involving development of algorithms capable of deriving grammars from a set of sample sentences has received paramount importance in recent times. A number of reports (see the first six references) have come out in the literature on the automated inference of regular, context free and context sensitive grammars in deterministic, nondeterministic and statistical domain. In recent years, with the growth of fuzzy set theory, fuzzy automata have also received considerable importance in the literature (see Hornung (1969) and Santos (1975)). Although the problem of designing a fuzzy automaton has been tackled by many researchers, the design of a fuzzy grammatical inference machine from sample strings, however, still remains to be investigated. An approach to the inference o f fuzzy regular grammar using formal power series has been presented in this paper. The results relating the theory of formal power series in noncommuting variable with the formal language theory were first reported by Schutzenberger (1961) and subsequently by Fliess (1974) and Salomaa and Soittola (1978). The inference technique has been applied to classify forty-five classes of Bengali alphabetic characters.
In this section some of the definitions and results of the theory of formal power series are introduced. Definition. Let M be a monoid and A be a semiring. The mappings r, of M into A, are called formal power series, given by
r = ~ (r, w) w.
(1)
w~M
The values of (r, w)~ A are also referred to as the coefficients of the series. The collection of all formal power series r, as defined above, is denoted by A ( ( M ) ) . Given any language L c_ VT~, the language may be uniquely associated with a formal power series r belonging to A ((V~)). For the purpose of inference of regular grammars, we would only require rational series, and recognizable series. An element r of A ((M)) is termed as quasiregular if (r, 2) = 0. The quasi-regular series has the property that the sequence r, r 2, r 3, ... converges to 0, and lim ~=~ r e exists. If r is quasiregular, then the series r = Y~k_-i rk is called a quasi-inverse of r. A subsemiring Q C A ( ( M ) ) is rationally closed
0167-8655/83/$3.00 © 1983, Elsevier Science Publishers B.V. (North-Holland)
27
Volume 2, Number 1
PATTERN RECOGNITION LETTERS
iff it contains the quasi-inverse of every quasiregular element. The family of A-rational series over M (in symbols Arat((M)) is the smallest rationally closed subset of A((M)), containing all polynomials, i.e. Arat((M)) CA((M)). A series r of A ((M)) is termed A-recognisable according to
reArec((M)) iff r = ( r , 2 ) + ~ p(~w)w,
(2)
w:~2
where t.t:M---~A re×m, m ¢ l is a representation. The value P(aij) can be expressed as linear combination of the entries aij with coefficients in A, i.e.
p(aii)= ~ aijPij, p i j e A .
(3)
I.J
Next we state Schutzenberger's theorem providing a convenient characterisation of recognisable power series. Theorem 1. I f reArec((M)), then there exist a row
vector a, a representation g and a column vector fl such that r= ~,, (a(lUW)fl)w.
October 1983
to A. Let us denote the set of all functions from V~ to A by A v;. Then obviously the sets A v; and A (( V~)> are, in a sense, equivalent. The set A v; also provides a convenient way of visualising the columns of H(r) as elements in A v7 It may be noted that with the column of H(r) corresponding to the word v ~ V~ (i.e. the oth column of H(r)), we may associate the function F v c A v~ as follows:
Fo(u) = (r, uo),
u ~ V~-.
(5)
In other words Fo(u) is essentially equivalent to the (u, o)th entry of H(r). The following theorem due to Fliess (1974) determines whether any given formal power series r is a rational series. Theorem 3. Let A be a commutative semiring and
reA<(V~)). Then the following conditions are equivalent: (i) r belongs t o Arat((V~)) (ii) The subsemimodules o f A v7 generated by the columns o f H(r) are contained in a finitely generated stable subsemimodule of AvT.
(4)
wEM
Conversely, any series
(a(uw)~)w w~M
In the case of inference of fuzzy grammar we will only be concerned with semiring [0, 1] with max min operation. Hereinafter this semiring will be denoted as fuzzy semiring.
belongs to Arec((M)). Theorem 2. (Kleene - Schutzenberger Theorem). For a free monoid V~- the families o f Arec((VT~)) and Arat((V~)) coincide.
The proofs of the above theorem may be found in Eilenberg (1974) and Salomaa and Soittola (1978). The rational power series as described, however, may be characterised by a Hankel matrix which is defined below. Definition. The Hankel matrix of r (r e A ((V~))) is a doubly infinite matrix H(r) whose rows and columns are indexed by the words of V~ and whose elements with indices u (row index) and o (column index) is equal to (r, uo).
According to the definition any formal power series r e A ((V~)) is essentially a mapping from Vfl 28
Inference of fuzzy regular grammar Definition. A fuzzy language over an alphabet Vr is defined to be a fuzzy set in V~ and a string x of V~ has a membership grade /~(x) [0 _
A finite fuzzy automaton over an
Volume 1, Number 1
PATTERN RECOGNITION
alphabet VT is a machine that accepts the regular fuzzy language and is given by
M =(Q, ~z, F, r/) where, (i) Q = {ql . . . . . qn} is a nonempty finite set of internal states. (ii) zr is an n-dimensional fuzzy row vector known as initial state designator, i.e. 2l" = (7/'ql . . . . .
LETTERS
October 1983
In this case we say that /] is dependent on { h 1. . . . . hn}. If no such ai (ale:O) exists, /~ is said to be independent of {hi . . . . . hn}. In view of the above observation regarding independent column, Corollary 1 follows directly from Theorem 3.
Corollary 1. I f A is a fuzzy semiring then r ~ A rat(( V~- )) iff there are finitely many independent columns o f H(r).
71"q.).
(iii) q = (r/q, . . . . . r/q,) is an n-dimensional column vector called the final state designator and (iv) F(x) is a fuzzy transition matrix s.t. for qi, q j e Q and x e Vr, the ( i - j ) t h element of F(x) is equal to fM(qi, Xj, qj) and the function
f M : Q × V T × Q ~ [ O , 1] is the fuzzy state transition function. This implies that for qi, q j e Q and x e V r, fM(qi, X, qj) is the grade of transition from state qi to state qj when the input is x. The grade of transition from one state q~ to another qm for an input string sequence of length m is defined by an m-ary fuzzy relation. The fuzzy state transition is given by
fM(ql, X, qm) = max q,eQ min [fM(ql, XI, q 2 ) ,
Assuming now H(r) has finitely m a n y independent columns and r e A rat (( V7~)), from Theorems 1 and 2, r may be expressed as r = ~ (a(pw)fl)w, where a is a row vector, fl is a column vector, and p is a representation and w e V~. Since H(r) has finitely many independent columns let Fo, ..... Fore constitute a minimal set of independent columns of H(r) associated with vl ..... Vm where oieV~, for i = l . . . . . m, are the strings associated with these columns. Now since Fo~. . . . . Fore constitute a finite set of independent columns of H(r), xF i ( x e Vr) must be linearly dependent on Fo~. . . . . Fore where x F / s h o u l d be interpreted as xF~(u)=Fui(ux), u 6 V~. Hence x ~ may be represented as
xF,,-= ~ (Px)ijFij for x e Vr, i-I
fM(q2, X2, q 3 ) . . . . . fM(qm - 1, Xm, qm)]
(6)
which is the membership grade of transition from state q~ to qm when x=X~Xz...Xm. Given a set o f sentences of finite length from a positive sample set of fuzzy language, the fuzzy Hankel matrix is formed using all the possible factorisations of each of the strings wi; the elements of the fuzzy Hankel matrix h(i,j) lie in the interval [0, 1]. Now given the fuzzy column vectors (h l . . . . . hn) (i.e. hi~.A n where A is a fuzzy semiring), a fuzzy column v e c t o r / ] is said to belong to the A fuzzy subsemimodule generated by the set of generators {h I. . . . . hn} if there exists a set of parameters al . . . . . an, not all a i = 0 and ale[O, 1] such that /~ = m a x [ m i n ( a l , hi), min(a2, h2), .... rain(an, hn)].
(8)
where p : V~ ~ A m×m
(7)
We will have to establish now that p is a representation, i.e. we show that J,/(W1W2) = p(wl)P(w2). Assuming (8) holds for x = w 1 and X = W2,
(Wlw2) Fo,(0 )=Fi(OW2)
where tS= OWl
= E (Uw2)Fi(O J J
J
= ~ (IdWll.lW2)kFok(O). k
(9)
Hence, this equation is valid for x = w I w2. Since the above equation is valid for x ~ VT, it also holds for any x ~ V~. Thus to construct /2, we need to consider the dependencies of xF/, for i = 1. . . . . m, and x ~ Vr on F~, ..... Fo,, . 29
Volume 1, Number 1
PATTERN RECOGNITION LETTERS
Once p is constructed, a and fl m a y be constructed as follows. Since r belongs to a finitely generated subsemimodule of A ((V/*)), there exist elements fll . . . . . tim ~ A , such that r = ~ fliFi where Fi is now treated as a function in A v,~. Then
(r, w) = ~ fliFi(w) = ~ fli(wFi)2 = 21~i 2 (]'lW)jiFi(~)
(10)
= (F 1(,~)F2 (J.) . . . . . Fm(2))pw(fll
.....
]~m) T
(where T stands for matrix transposition).
a = tFl()O... Fm(J.)].
(11)
So a corresponds to the entries in FI . . . . . Fm for the row in H(r) labelled by ~.e V~. Since F ~ ( w ) = (r, w ) , f r o m (10) fli corresponds to the coefficients o f F~ in the expansion o f Fa in terms o f (F 1. . . . . Fro). We can now construct a minimal fuzzy a u t o m a t a that accepts the sample strings as follows. The n u m b e r o f states in the a u t o m a t a would be equal to the n u m b e r o f independent columns o f the H a n k e l matrix and we identify the states o f the fuzzy a u t o m a t a with the independent columns o f the H a n k e l matrix. We associate a with n, i.e. the initial states, fl with r/, i.e. final states, and p(x) is used to define the state transition function for the a u t o m a t a .
Application
The m e t h o d o f a u t o m a t e d inference o f fuzzy
October 1983
g r a m m a r has been applied to classify forty five classes o f Bengali alphabetic characters. The pattern classification p r o b l e m has been d e c o m p o s e d in two phases. Using fuzzy similarity criterion, hierarchical binary decision trees have been constructed in the first stage (see Biswas). Pattern classes belonging to the leaves o f the decision tree f o r m a set o f subclasses which cannot be classified in the first stage o f classification (see Biswas). These pattern classes n o w become candidates for linguistic analysis and are fed to the structural classifier in the second phase. Each pattern has been encoded in the f o r m o f strings o f qualifiers over Vr = (a, b, c, d), where a, b, c and d indicate the four scanning directions. The encoding has been done by tracking the b o u n d a r y o f the pattern in clockwise direction. Linguistic analysis has been applied to a very small zone o f the pattern string, where the structural dissimilarily between the pattern classes is m a x i m u m . All the strings o f a particular pattern class are next associated with a generative g r a m m a r which is not k n o w n a priori. The g r a m m a r s have been constructed using the inference procedure described above. The fuzzy membership function o f a particular string in a pattern class has been determined f r o m its frequency o f occurance. Let us consider a positive sample set R ÷ = {0.8ab, 0.8aabb, 0.3abc, 0.2bc, 0.9abbc} corresponding to a pattern class. The finite submatrix o f the fuzzy Hankel matrix is shown in Table 1. The set o f independent columns o f the fuzzy Hankel matrix has been indicated as
Table 1 Sample set R + = {0.8ab, 0.8aabb, 0.3abc, 0.2bc, 0.9abbc}. The finite submatrix of the fuzzy Hankel matrix H(r)
H(r)
2.
ab
b
abc
bc
c
abbc
bbc
aabb
abb
bb
1 a
0 0
0.8 0
0 0.8
0.3 0
0.2 0.3
0 0
0.9 0
0 0.9
0.8 0
0 0.8
0 0
ab abc bc abb abbc aa aab aabb
0.8 0.3 0 0.9
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0.9 0 0 0 0
0.3 0 0 0.9 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0
0 0
0 0.8
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0.8 0
0.8
0
0
0
0
0
0
0
0
0
0
El
F2
F5
F6
F7
30
0.2
Volume 2, Number 1
PATTERN R E C O G N I T I O N LETTERS
October 1983
/=1, F2, Fs, F6 and F7. The matrices p(x) for
x={a, b, c}
are obtained by observing the dependencies of xF/, Fie {F1,F2, Fs, F6, F7} in Santos (1975). These matrices are given in Table 2. The a and/3 vectors are next computed from (10) and
0..8
(11): a=[0
0.9
0.2
0
0
0
0],
1~=[1
0
0
0
0
0
01.
$7
The minimal fuzzy automaton, constructed from the matrices/z(a), Ia(b) and p(c) and the vectors a and ,6, is shown in Fig. 1. Once a finite fuzzy automaton that accepts R + is identified, a fuzzy regular grammar G such that language generated by G is equal to R +, can be easily constructed from the description of the identified fuzzy automaton (Santos (1975)). The grammar G corresponding to the finite fuzzy automaton identified here is also shown in Table 3. The grammar corresponding to each class of patterns is constructed using the inference procedure as described above.
Fig. 1. Fuzzy automaton.
Table 3
Fuzzy regular grammar G = ( V r , VN, $2, P) with V r = { a , b , c } , VN={$2,$3,84,$5,36,$7} and p.. $2°~8aS7 $31-~bS4 $61bS7 820.~ aS 3 84 I, c $7 1 b $2 ~ aS5
S 5 L bS 3 Ss ~ aS6
Conclusion Table 2
Matrices p(a), lt(b) and p(c) for • = {a,b,c} #(a)
Sl
S2
S3
S4
S5
S6
87
S1 $2 S3 $4 Ss $6 $7
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0.3 0 0 0 0 0
0 0 0 0 0 0 0
0 1 0 0 0 0 0
0 0 0 0 0.8 0 0
0 0.8 0 0 0 0 0
p(b)
$1
$2
$3
$4
$5
$6
Sv
S1 S2 S3 $4 Ss $6 $7
0 0 0 0 0 0 1
0 0 0 0 0 0 0
0 0 0 0 1 0 0
0 0 1 0 0 0 0
0 1 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 1 0
u(c)
Sl
$2
$3
$4
Ss
$6
$7
S1 S2 $3 $4 Ss $6 $7
0 0 0 1 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
An automated inference of fuzzy regular grammar has been described here. The methodology offers a new approach to the process of inference of fuzzy grammars. The method may be used in the syntactic classification of patterns, where the patterns may be encoded in the form of strings.
References Biermann, A.W. and J.A. Feldman (1972). On the synthesis of finite state machine from samples of their behaviour. IEEE Trans. Computer 21, 592-597.
Biswas, P. A fuzzy hybrid model for pattern classification with application to recognition of handprinted Devanagari, Ph.D. Dissertation, School of Computer and Systems Science, J.N.U., India. Chomsky, W. Syntactic structures, Mouton, The Hague, The Netherlands. Eilenberg, S. (1974). Automata, Languages and Machines, Vol. A. Academic Press, New York. Feldman, J.A. (1967). First thoughts on grammatical inference, Artificial Intelligence Memo 55, Computer Science Dept., Stanford University, Stanford, CA. Fliess, M. (1974). Sur divers produits de series formelles. Bull. Soc. Math. France 102, 181-191.
31
Volume 1, Number 2
PATTERN RECOGNITION LETTERS
Horning, J.J. (1969). A study of grammatical inference. Tech. Rep. C.S.-139, Computer Science Dept., Stanford University, Stanford, CA. Santos, E.S. (1974). Fuzzy automata and languages, US-Japan Seminar on Fuzzy sets and their applications, Berkeley, CA. Santos, E.S. (1974). Fuzzy automata and languages, US-Japan Seminar on Fuzzy sets and their applications, Berkeley, CA.
32
October 1983
Schutzenberger, M.P. (1961). On definition of a family of automata, Information and Control 4, 245-270. Salomaa, A. and M. Soittola (1978). Automata Theoretic Aspects o f Formal Power Series, Springer, New York. Thomason, M.G. and P.N. Marinos (1974). Deterministic acceptors of regular 'fuzzy languages. IEEE Trans. Systems Man Cybernet. 4, 228-230.
October 1983
Inference of fuzzy regular pattern grammar A.K. MAJUMDAR
and A.K. ROY
Indian Institute o f Technology, Kharagpur, India-721302 Received 7 February 1983 Revised 30 April 1983
Abstract: This paper contains an approach to the inference of fuzzy pattern grammar from a set of sample strings. The grammar is inferred from a finite submatrix of the fuzzy Hankel matrix of the coefficients of formal power series representing the set of sample strings from each pattern class. Key words and phrases: Inference, formal power series, pattern grammar, Hankel matrix, fuzzy grammar.
Introduction
Preliminaries of formal power series
The problem of grammatical inference involving development of algorithms capable of deriving grammars from a set of sample sentences has received paramount importance in recent times. A number of reports (see the first six references) have come out in the literature on the automated inference of regular, context free and context sensitive grammars in deterministic, nondeterministic and statistical domain. In recent years, with the growth of fuzzy set theory, fuzzy automata have also received considerable importance in the literature (see Hornung (1969) and Santos (1975)). Although the problem of designing a fuzzy automaton has been tackled by many researchers, the design of a fuzzy grammatical inference machine from sample strings, however, still remains to be investigated. An approach to the inference o f fuzzy regular grammar using formal power series has been presented in this paper. The results relating the theory of formal power series in noncommuting variable with the formal language theory were first reported by Schutzenberger (1961) and subsequently by Fliess (1974) and Salomaa and Soittola (1978). The inference technique has been applied to classify forty-five classes of Bengali alphabetic characters.
In this section some of the definitions and results of the theory of formal power series are introduced. Definition. Let M be a monoid and A be a semiring. The mappings r, of M into A, are called formal power series, given by
r = ~ (r, w) w.
(1)
w~M
The values of (r, w)~ A are also referred to as the coefficients of the series. The collection of all formal power series r, as defined above, is denoted by A ( ( M ) ) . Given any language L c_ VT~, the language may be uniquely associated with a formal power series r belonging to A ((V~)). For the purpose of inference of regular grammars, we would only require rational series, and recognizable series. An element r of A ((M)) is termed as quasiregular if (r, 2) = 0. The quasi-regular series has the property that the sequence r, r 2, r 3, ... converges to 0, and lim ~=~ r e exists. If r is quasiregular, then the series r = Y~k_-i rk is called a quasi-inverse of r. A subsemiring Q C A ( ( M ) ) is rationally closed
0167-8655/83/$3.00 © 1983, Elsevier Science Publishers B.V. (North-Holland)
27
Volume 2, Number 1
PATTERN RECOGNITION LETTERS
iff it contains the quasi-inverse of every quasiregular element. The family of A-rational series over M (in symbols Arat((M)) is the smallest rationally closed subset of A((M)), containing all polynomials, i.e. Arat((M)) CA((M)). A series r of A ((M)) is termed A-recognisable according to
reArec((M)) iff r = ( r , 2 ) + ~ p(~w)w,
(2)
w:~2
where t.t:M---~A re×m, m ¢ l is a representation. The value P(aij) can be expressed as linear combination of the entries aij with coefficients in A, i.e.
p(aii)= ~ aijPij, p i j e A .
(3)
I.J
Next we state Schutzenberger's theorem providing a convenient characterisation of recognisable power series. Theorem 1. I f reArec((M)), then there exist a row
vector a, a representation g and a column vector fl such that r= ~,, (a(lUW)fl)w.
October 1983
to A. Let us denote the set of all functions from V~ to A by A v;. Then obviously the sets A v; and A (( V~)> are, in a sense, equivalent. The set A v; also provides a convenient way of visualising the columns of H(r) as elements in A v7 It may be noted that with the column of H(r) corresponding to the word v ~ V~ (i.e. the oth column of H(r)), we may associate the function F v c A v~ as follows:
Fo(u) = (r, uo),
u ~ V~-.
(5)
In other words Fo(u) is essentially equivalent to the (u, o)th entry of H(r). The following theorem due to Fliess (1974) determines whether any given formal power series r is a rational series. Theorem 3. Let A be a commutative semiring and
reA<(V~)). Then the following conditions are equivalent: (i) r belongs t o Arat((V~)) (ii) The subsemimodules o f A v7 generated by the columns o f H(r) are contained in a finitely generated stable subsemimodule of AvT.
(4)
wEM
Conversely, any series
(a(uw)~)w w~M
In the case of inference of fuzzy grammar we will only be concerned with semiring [0, 1] with max min operation. Hereinafter this semiring will be denoted as fuzzy semiring.
belongs to Arec((M)). Theorem 2. (Kleene - Schutzenberger Theorem). For a free monoid V~- the families o f Arec((VT~)) and Arat((V~)) coincide.
The proofs of the above theorem may be found in Eilenberg (1974) and Salomaa and Soittola (1978). The rational power series as described, however, may be characterised by a Hankel matrix which is defined below. Definition. The Hankel matrix of r (r e A ((V~))) is a doubly infinite matrix H(r) whose rows and columns are indexed by the words of V~ and whose elements with indices u (row index) and o (column index) is equal to (r, uo).
According to the definition any formal power series r e A ((V~)) is essentially a mapping from Vfl 28
Inference of fuzzy regular grammar Definition. A fuzzy language over an alphabet Vr is defined to be a fuzzy set in V~ and a string x of V~ has a membership grade /~(x) [0 _
A finite fuzzy automaton over an
Volume 1, Number 1
PATTERN RECOGNITION
alphabet VT is a machine that accepts the regular fuzzy language and is given by
M =(Q, ~z, F, r/) where, (i) Q = {ql . . . . . qn} is a nonempty finite set of internal states. (ii) zr is an n-dimensional fuzzy row vector known as initial state designator, i.e. 2l" = (7/'ql . . . . .
LETTERS
October 1983
In this case we say that /] is dependent on { h 1. . . . . hn}. If no such ai (ale:O) exists, /~ is said to be independent of {hi . . . . . hn}. In view of the above observation regarding independent column, Corollary 1 follows directly from Theorem 3.
Corollary 1. I f A is a fuzzy semiring then r ~ A rat(( V~- )) iff there are finitely many independent columns o f H(r).
71"q.).
(iii) q = (r/q, . . . . . r/q,) is an n-dimensional column vector called the final state designator and (iv) F(x) is a fuzzy transition matrix s.t. for qi, q j e Q and x e Vr, the ( i - j ) t h element of F(x) is equal to fM(qi, Xj, qj) and the function
f M : Q × V T × Q ~ [ O , 1] is the fuzzy state transition function. This implies that for qi, q j e Q and x e V r, fM(qi, X, qj) is the grade of transition from state qi to state qj when the input is x. The grade of transition from one state q~ to another qm for an input string sequence of length m is defined by an m-ary fuzzy relation. The fuzzy state transition is given by
fM(ql, X, qm) = max q,eQ min [fM(ql, XI, q 2 ) ,
Assuming now H(r) has finitely m a n y independent columns and r e A rat (( V7~)), from Theorems 1 and 2, r may be expressed as r = ~ (a(pw)fl)w, where a is a row vector, fl is a column vector, and p is a representation and w e V~. Since H(r) has finitely many independent columns let Fo, ..... Fore constitute a minimal set of independent columns of H(r) associated with vl ..... Vm where oieV~, for i = l . . . . . m, are the strings associated with these columns. Now since Fo~. . . . . Fore constitute a finite set of independent columns of H(r), xF i ( x e Vr) must be linearly dependent on Fo~. . . . . Fore where x F / s h o u l d be interpreted as xF~(u)=Fui(ux), u 6 V~. Hence x ~ may be represented as
xF,,-= ~ (Px)ijFij for x e Vr, i-I
fM(q2, X2, q 3 ) . . . . . fM(qm - 1, Xm, qm)]
(6)
which is the membership grade of transition from state q~ to qm when x=X~Xz...Xm. Given a set o f sentences of finite length from a positive sample set of fuzzy language, the fuzzy Hankel matrix is formed using all the possible factorisations of each of the strings wi; the elements of the fuzzy Hankel matrix h(i,j) lie in the interval [0, 1]. Now given the fuzzy column vectors (h l . . . . . hn) (i.e. hi~.A n where A is a fuzzy semiring), a fuzzy column v e c t o r / ] is said to belong to the A fuzzy subsemimodule generated by the set of generators {h I. . . . . hn} if there exists a set of parameters al . . . . . an, not all a i = 0 and ale[O, 1] such that /~ = m a x [ m i n ( a l , hi), min(a2, h2), .... rain(an, hn)].
(8)
where p : V~ ~ A m×m
(7)
We will have to establish now that p is a representation, i.e. we show that J,/(W1W2) = p(wl)P(w2). Assuming (8) holds for x = w 1 and X = W2,
(Wlw2) Fo,(0 )=Fi(OW2)
where tS= OWl
= E (Uw2)Fi(O J J
J
= ~ (IdWll.lW2)kFok(O). k
(9)
Hence, this equation is valid for x = w I w2. Since the above equation is valid for x ~ VT, it also holds for any x ~ V~. Thus to construct /2, we need to consider the dependencies of xF/, for i = 1. . . . . m, and x ~ Vr on F~, ..... Fo,, . 29
Volume 1, Number 1
PATTERN RECOGNITION LETTERS
Once p is constructed, a and fl m a y be constructed as follows. Since r belongs to a finitely generated subsemimodule of A ((V/*)), there exist elements fll . . . . . tim ~ A , such that r = ~ fliFi where Fi is now treated as a function in A v,~. Then
(r, w) = ~ fliFi(w) = ~ fli(wFi)2 = 21~i 2 (]'lW)jiFi(~)
(10)
= (F 1(,~)F2 (J.) . . . . . Fm(2))pw(fll
.....
]~m) T
(where T stands for matrix transposition).
a = tFl()O... Fm(J.)].
(11)
So a corresponds to the entries in FI . . . . . Fm for the row in H(r) labelled by ~.e V~. Since F ~ ( w ) = (r, w ) , f r o m (10) fli corresponds to the coefficients o f F~ in the expansion o f Fa in terms o f (F 1. . . . . Fro). We can now construct a minimal fuzzy a u t o m a t a that accepts the sample strings as follows. The n u m b e r o f states in the a u t o m a t a would be equal to the n u m b e r o f independent columns o f the H a n k e l matrix and we identify the states o f the fuzzy a u t o m a t a with the independent columns o f the H a n k e l matrix. We associate a with n, i.e. the initial states, fl with r/, i.e. final states, and p(x) is used to define the state transition function for the a u t o m a t a .
Application
The m e t h o d o f a u t o m a t e d inference o f fuzzy
October 1983
g r a m m a r has been applied to classify forty five classes o f Bengali alphabetic characters. The pattern classification p r o b l e m has been d e c o m p o s e d in two phases. Using fuzzy similarity criterion, hierarchical binary decision trees have been constructed in the first stage (see Biswas). Pattern classes belonging to the leaves o f the decision tree f o r m a set o f subclasses which cannot be classified in the first stage o f classification (see Biswas). These pattern classes n o w become candidates for linguistic analysis and are fed to the structural classifier in the second phase. Each pattern has been encoded in the f o r m o f strings o f qualifiers over Vr = (a, b, c, d), where a, b, c and d indicate the four scanning directions. The encoding has been done by tracking the b o u n d a r y o f the pattern in clockwise direction. Linguistic analysis has been applied to a very small zone o f the pattern string, where the structural dissimilarily between the pattern classes is m a x i m u m . All the strings o f a particular pattern class are next associated with a generative g r a m m a r which is not k n o w n a priori. The g r a m m a r s have been constructed using the inference procedure described above. The fuzzy membership function o f a particular string in a pattern class has been determined f r o m its frequency o f occurance. Let us consider a positive sample set R ÷ = {0.8ab, 0.8aabb, 0.3abc, 0.2bc, 0.9abbc} corresponding to a pattern class. The finite submatrix o f the fuzzy Hankel matrix is shown in Table 1. The set o f independent columns o f the fuzzy Hankel matrix has been indicated as
Table 1 Sample set R + = {0.8ab, 0.8aabb, 0.3abc, 0.2bc, 0.9abbc}. The finite submatrix of the fuzzy Hankel matrix H(r)
H(r)
2.
ab
b
abc
bc
c
abbc
bbc
aabb
abb
bb
1 a
0 0
0.8 0
0 0.8
0.3 0
0.2 0.3
0 0
0.9 0
0 0.9
0.8 0
0 0.8
0 0
ab abc bc abb abbc aa aab aabb
0.8 0.3 0 0.9
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0.9 0 0 0 0
0.3 0 0 0.9 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0
0 0
0 0.8
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0.8 0
0.8
0
0
0
0
0
0
0
0
0
0
El
F2
F5
F6
F7
30
0.2
Volume 2, Number 1
PATTERN R E C O G N I T I O N LETTERS
October 1983
/=1, F2, Fs, F6 and F7. The matrices p(x) for
x={a, b, c}
are obtained by observing the dependencies of xF/, Fie {F1,F2, Fs, F6, F7} in Santos (1975). These matrices are given in Table 2. The a and/3 vectors are next computed from (10) and
0..8
(11): a=[0
0.9
0.2
0
0
0
0],
1~=[1
0
0
0
0
0
01.
$7
The minimal fuzzy automaton, constructed from the matrices/z(a), Ia(b) and p(c) and the vectors a and ,6, is shown in Fig. 1. Once a finite fuzzy automaton that accepts R + is identified, a fuzzy regular grammar G such that language generated by G is equal to R +, can be easily constructed from the description of the identified fuzzy automaton (Santos (1975)). The grammar G corresponding to the finite fuzzy automaton identified here is also shown in Table 3. The grammar corresponding to each class of patterns is constructed using the inference procedure as described above.
Fig. 1. Fuzzy automaton.
Table 3
Fuzzy regular grammar G = ( V r , VN, $2, P) with V r = { a , b , c } , VN={$2,$3,84,$5,36,$7} and p.. $2°~8aS7 $31-~bS4 $61bS7 820.~ aS 3 84 I, c $7 1 b $2 ~ aS5
S 5 L bS 3 Ss ~ aS6
Conclusion Table 2
Matrices p(a), lt(b) and p(c) for • = {a,b,c} #(a)
Sl
S2
S3
S4
S5
S6
87
S1 $2 S3 $4 Ss $6 $7
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0.3 0 0 0 0 0
0 0 0 0 0 0 0
0 1 0 0 0 0 0
0 0 0 0 0.8 0 0
0 0.8 0 0 0 0 0
p(b)
$1
$2
$3
$4
$5
$6
Sv
S1 S2 S3 $4 Ss $6 $7
0 0 0 0 0 0 1
0 0 0 0 0 0 0
0 0 0 0 1 0 0
0 0 1 0 0 0 0
0 1 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 1 0
u(c)
Sl
$2
$3
$4
Ss
$6
$7
S1 S2 $3 $4 Ss $6 $7
0 0 0 1 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
An automated inference of fuzzy regular grammar has been described here. The methodology offers a new approach to the process of inference of fuzzy grammars. The method may be used in the syntactic classification of patterns, where the patterns may be encoded in the form of strings.
References Biermann, A.W. and J.A. Feldman (1972). On the synthesis of finite state machine from samples of their behaviour. IEEE Trans. Computer 21, 592-597.
Biswas, P. A fuzzy hybrid model for pattern classification with application to recognition of handprinted Devanagari, Ph.D. Dissertation, School of Computer and Systems Science, J.N.U., India. Chomsky, W. Syntactic structures, Mouton, The Hague, The Netherlands. Eilenberg, S. (1974). Automata, Languages and Machines, Vol. A. Academic Press, New York. Feldman, J.A. (1967). First thoughts on grammatical inference, Artificial Intelligence Memo 55, Computer Science Dept., Stanford University, Stanford, CA. Fliess, M. (1974). Sur divers produits de series formelles. Bull. Soc. Math. France 102, 181-191.
31
Volume 1, Number 2
PATTERN RECOGNITION LETTERS
Horning, J.J. (1969). A study of grammatical inference. Tech. Rep. C.S.-139, Computer Science Dept., Stanford University, Stanford, CA. Santos, E.S. (1974). Fuzzy automata and languages, US-Japan Seminar on Fuzzy sets and their applications, Berkeley, CA. Santos, E.S. (1974). Fuzzy automata and languages, US-Japan Seminar on Fuzzy sets and their applications, Berkeley, CA.
32
October 1983
Schutzenberger, M.P. (1961). On definition of a family of automata, Information and Control 4, 245-270. Salomaa, A. and M. Soittola (1978). Automata Theoretic Aspects o f Formal Power Series, Springer, New York. Thomason, M.G. and P.N. Marinos (1974). Deterministic acceptors of regular 'fuzzy languages. IEEE Trans. Systems Man Cybernet. 4, 228-230.