The stability of a model of recognition algorithms

U.S.S.R. Comput.Maths.Math.Phys.Vol.23,No.l,pp,133-137,1983 Printed in Great Britain

OO41-5553/83 $iO.OO+O.O0 9 1984 Pergamon Press Ltd.

THE STABILITY OF A MODEL OF RECOGNITION ALGORITHMS* s.I. KASHKEVICH

and V.V. KRASNOPROSHIN

A family of recognition algorithms with generalized standards is investigated, the validity of its linear closure is proved. A concept of algorithm stability recognition problems is introduced, and the sufficient conditions for stability given. It is shown that a correct algorithm belonging to the family considered stable. Section Let

and in are is

i. |

{D}=

U K~

be a set of an arbitrary nature, whose elements we shall call admissible objects; the subsets , Kj, ]=|, 2,..., I will be referred to as classes. The description S = S(D) is known for each admissible object D~{D}. Throughout, we identify the objects by their descriptions and instead of set {D} we examine set {S} of the admissible descriptions broken down into I classes. Suppose that the initial information 10 about classes K,,..,,K, is known, and a finite subset (sample) Sac{S}, S ~= {S',..., S q} is separated. Then the recognition problem can be formulated as follows: on the basis of the initial information J0 we wish to determine the membership of each object from S q in the classes Kj, ]=|, 2,..., I. We denote this problera by Z={/0, Sq}. The universal set of problems Z for all possible J~, S 7 forms the set {Z}. we complete our description with the definitions and assertions from /i/." Letl,(A} be a set of recognition algorithms. Then A ( Z ) = ll~iJ[[ax,,~la~{0, |, A}, i=I, 2 .... , q, ]= I, 2 ..... where ~ 0 = ~ ( A ) is the value of the predicate P.,(Si)=,,Si~Af.~"computed by algorithm .4, and A is the failure of the algorithm to compute the predicate. The matrix ]]c%~]]~x,, where ~ = P j ( S ~) is described as an information matrix of the subset S~, and in this case the row a(S~)~(~,,...~ ~,t) is known as an inf6rmation vector of the obj~.ct Si, i=|, 2,..., q. Algorithm A is correct if A(Z)=I[~[[q• , and the family of algorithms, in which for each problem Z~{Z} at least one correct algorithm exists, is correct on {Z}. Any recognition algorithm A can be expressed as .4=B.C, where B(Z)=][a~ll~• C(![a~I[)= ll~i~Uq• and ai~=a~j(B) are real numbers. Usually, /] is a recognition operator, and C is a decision rule of the algorithm A. Thus, the set {M} generates sets (B} and {C} respectively. The decision rule C~{C} is correct on {S} if for any finite subset Sq~{S} at least one numerical matrix [[aJlq• exists such thatC(Ua~]])=II=~ll~• For example, the threshold decision rule C=C(60, 6~) where

f,

I

A,

a,~>6,, 6o~a,j6,,

(i) i=I,2 .....q, ]=1,2 .....l,

is correct. Throughout, only correct decision rules are discussed. Let us introduce into {B} the addition and multiplication by a scalar as a component-bycomponent operation on the elements of the corresponding matrices IIaoH. %~nese operators generate the linear closure L(B). Consider L(A)= {A=B'.C]B'~L(B), C~{C}}. Problem Z is called complete with respect to {B} if the set of matrices {B(Z)IB~{B}} contains the basis in the space of numerical qXl, me{rices. Theorem I. If the set {Z} consists of problems linear closure L(.4) on {Z} is correct.

complete with respect to

{B}, then the

Section 2. we will now describe a model of recognition algorithms with so-called generalized standards. Separate algorithms of this class were dealt with in /2/ and /3/. The feature of such algorithms is their conversion of the initial information w i t h e view of reducing its volume; they are very convenient when solving problems with a large volume of initial data since they reduce the demands on computer resources. A particular family of the generalized-standard algorithms is given by successively performing the following stages (.for simplicity, we assume q=l, and therefore the subset S ~ will consist of a single element S). S t a g e i. Let ~ be a set of arbitrary nature, and {[0} a set of the initial information on breaking down {S} into classes. We introduce the function qt: { I 0 } ~ ~, where qt(]o)= (M, ..... ~]It),~[j~, ]=|, 2 .... , I 9 and refer to the function ~, as a generalization operator,and *Zh.vych.Mat.i

mat.Fiz.,Vol.23,No.l,pp.191-197,1983 133

134 call Mj

a generalized

standard of class

Ki.

Example i. Let {S} be a metric space with distance o. The initial information I0 is given in standard form (see /i/), to= {S,..... S~, a(S,) ..... a(S~)}, where Si~{S}, i=|, 2 ..... re. We assume that KtflKj=Z, te6]. Then we can consider, without loss of generality, that S~j_I+,,... , S=jbelongs to the class Kj, j=|, 2 .... , /.,where O=mo<...
3tj

--=~

t

Inj--mj-t

S,,

j = l , 2 .... l.

f-m/.t4-!

Example 2. Let V=(v,,..., v~) be a certain alphabet, ~-,(V) a set of words of length n, comprising letters from V, and r~(T) the k-th symbol, k<.n, of the word T~Y,(V). Furthermore let {S}=~,(V), and let the initial information be la={G, ..... G,}, where Gj is a grammar using alphabet V. In this case ,Kj=Gj"(~)NZ,(V), where Gj"(|I) is a set of words of length n, generated by the gram.~ar Gj, j=1, 2 ..... I. We assume ~=Za(r), where V = VU{A} anti ~Lq~V. The function q, can be given by the rule M~=(o,~,...,o,$), a ~ F , while old'V, o,~=v, if the proportion of the words of length n, obtained using the grammar G~, in which v occurs in the i-th place, l=|, 2,..., n, is not less than a certain threshold b, 0
a)

Then

the

,p2(S,M~)----II, if p(S,M~)<~j, t 0 otherwise

~,(S,M~)=

b) where

"h

'fl>O,

i+p' (s, ~t,) '

1=1,2 .....

l,

~j is a parameter.

Example 4. The conditions Mj) =(~,J, ....~,~), where

are as in Example 2.

~{j=[t,

(It~.~ V

if

~o

and

Assuraing

~=({0, I})", we have

(~2(S,

rt(5')=(if i,

otherwise

S t a g e I I I . we will determine the population of the functions W3;, j=|,2,...,l. Each of the functions represents ~t in the set of real numbers B. We write Fj(S)=q,~(q..(S,.]It),..., ~:(S, i~Ii)) and call Fj(S) the estimate of object S with respect to class Kj.

Example 5. If the function ~t is given as in Example found by one of the following rules: a)

F~(S) =b(S)r

b,

r.(S)=b(S)~z/r162

3a, the quantity

rj(s) can be

M,); j = t , 2 . . . . . l,

h ~ t , 2 . . . . . l, where

b, i=i, 2,...,n

Example 6.

are parameters.

Let the conditions

of Example

4 be satisfied and let

F, (Si := ~ b,~,', f--!

where

b~, i = | , 2 , . . . , n

are parameters.

Stages I - I I l describe the recognition operator B, i.e. B ( Z ) = HV,(S,)ll,• Joining to the operator B the fixed correct decision rule C we obtain a recognition algorithm with generalized standards. Let us consider a parametric family of recognition algorithms with generalized standards and analyze the question of the correctness of its linear closure in the set {Z} of problems Z={[o, Sq}. Let {S}=• n be a metric space with distance p and let the initial information I0 be given in standard form. Consider the set of recognition operators {~}={~(~,~,~0,~i)}, given in accordance with the rule :

fa={S},

M,

I

/'n,..t--m J

=~, S,, r--ml.t+!

/=1,2 .....

l;

135 ~ = B + is a set of positive numbers, and

i+p'(S',M~)'

q~,(S',M,)=

I',(~')=b,[~,'r h=l, 2,...,/,

i = 1 , 2 . . . . . q,

] = 1 , 2 . . . . . l;

~ ~:~(S',M~)], ] = 1 , 2 . . . . . l,

i=l, 2,...,q.

The family of recognition operators described, {~(~, ~, ~o, ~t)}~ depends on the parameters ~ = ( ' f . . . . . . "~,), ~ = ( b . . . . . . b,), . ~ ~ ..... xt~ (x, t . . . . . x,'). We define a set of recognition algorithms {.~} so that any algorithm ./T~{.~}, f[=~.C, where

~

{~}.

T h e o r e m 2. The linear clsoure L(2~) of the set of recognition algorithms with generalized standards {~[} is correct in {Z}. Proof. By Theorem 1 it is sufficient to prove that the set {Z} consists of problems complete with respect to {~}. We construct the recognition operators ~ , ~ t ~ } , tt=1,2,...,q, v=I,2, .... l, such that ~(Z)=ila,~ll~x,,where {~, if 0, i f

a(~=a~

It is obvious from a simple check ]=I, 2.....l and

~=,,, ]=u,

i*/=tZ,]#v, i=1,2 .... ,q, ]----I,2,...,/. that such an operator will be defined if we put x~~

t {i for ]=u, { I for i=u, x~= b,= 0 for ]~u, ]=I,2 .....Z, 0 for i=#u, i=|,2 .....q. { (I+p,(S~, M,)) for ]=o, ~i=

i

for ]~u,

]=1,2 .....I.

Thus the set of matrices {~v(Z)}, u=l, 2..... q, v=|~2 ..... l contains the basis in the space of numerical matrices qX[, and therefore the set of problems {Z} is complete with respect to { B } . This proves the theorem. We will denote the correct algorithm for problem Z={/0, E q} by A'=A'(Io, SO, A'~L(2[), and the recognition operator of this algorithm by B' S e c t i o n 4. Introducing the necessary concepts and definitions we shall now analyze the stability of the correct algorithm A'([o, S q) when the subset S q varies under the conditions imposed on the problem Z=(1o, Sq). Throughout, without loss of generality, the term 'sample' will be understood to mean a finite, and in some respects, ordered, s~nple from {S}. Let {S} be a metric with the distance p,and p(S,,S2) be the distance between objects St and S, in {S}. If ,.~'={S,',.... Sh'}, S"={S,", ....SA"} are two ordered samples of identical capacity, we shall describe tne distance between samples S' and S" as p (S', ,.~") = m a x p (S,', ,9,") The set B,(S')={S"lp(S',S") 0 exists such that for any sample /~'=/{,(Sq) the inequalities A(Io,SO=4(Io, D O hold. The recognition operator B is completely stable if for any sample D*EB,(B)(S~), where t B ~ q e(B)>0, the relation ]~ ,j()--ao(B)|<~p(D,S)[o(B), i=1,2 ..... q, ]=1,2 ..... l holds. Here B(10,/*) = ]]a~jlIq• B(lo, DO=IIa,/II~• and 0 < ~ o < + ~ for all i,]. We willcall the quantity e(B) the radius of stability of the operator /9. T h e o r e m 3. L~t the algorithm A=B.C, be given for solving the problem Z={/0, Sq}, and let B denote a completely stable operator with radius of stability ~(B) and C=C(60,6,). If ~i~(A)~A for all values of i and ], i=1,2,...,q,]=|,2 .... ,I then algorithm A is stable.

Proof.

Consider two sets of indices:

E~={(i,])]~o=~,

i = 1 , 2 . . . . . q, ]=1~'2 . . . . . l},

i~=OVt:

By the conditions of the theorem, EoUE,={I, 2,..., q}X{l, 2 ..... I}. Now let B(Io,S ~)=ua,~Lx, For the threshold values (i) of 60 and 6t we compute C o = rain (6~--a,,), Clearly,

o0>0, 0,>0.

We put

9 { e ( B ) , ~o .,

e=mm Sin~e

o, = rain (a,,--6,). (Ij)~z,

(~d) ezo

,

where

M=max~0(B L$

).

~lj(B)>0 for all i and ], we have e>0. Consider an arbitrary sample Dq~/{.(Sq). B is a completely stable operator, and since e~e(B), we have[ao'--aljl<~p(,~*,D~)M<~e2~l. Then, considering the choice of ~, we can say that |ao'--ao[<(~o for (i,])~Eo and [al/--a~jl
136 Let

L(B, . . . . . Bh)

Lemma or

B~L(Bt

be a linear closure of the recognition

operators

Bt,...,Bh.

i. If the recognition operators Bi ....,Bh are completely stable, .....B^) is also completely stable, and its stability radius is

then any operat-

e (B) = m i n e (B,). t ~f,~h

Proof.

By the definition of linear closure in /i/,

B = ~ a ~,,B~. We put

g ~ min s (B,) and analyze the arbitrary sample q, ]=|, 2 ..... I. Then

/J~B,(~q).

B(I,, S,) =Jla.l[ .... B(Io. 179 =lia,/ll,x. i=t, 2 . . . . .

Now let

h

la,,'-a,~l < Z IX,I la,/ (B,)-a,,CB,)I~
f-i

We select

t--I

By the foregoing inequality, lau'(B)--ao(B)[~
A'(/0,Sq),

appropriate

to problem

Z, is stable

(under the

Proof.

First we will show that the recognition operator B" is completely stable. ~ > 0 be an arbitrary number. We consider the sampling B(10,Sq)=[la0Hqx~, B(I0,/~q)= [lau'[lqx,, and assume that ~q~H,(S') where B~-{B}. By Theorem 2, L(/~) oontains base operators ~uc. With regard tothelr explicit form, for S u ~ S r and D ~ D , we have Let

a~j(B~,)={ [I+P'(6~'M~)I/II + Potherwise '(D='0

~/')|'

is,t,

]=v,

We will now estimate the quantity l'ao'(J~,,)-a,,(B~,)l for all possible admissible values of the indices i, ], u, and ~. Since ao' and a,j are given, it is sufficient to consider the case where i==, ]=v, u=i,2 .... ,q, and v=|,2,...,l. I. Let p(S~,M,)>0, we have

m.)-p' (D~, m._), < I= I p' (s~'l+pt (DL M,) - I

[a..' (/L,)

[p(SLM,)+p(D ~, M,) ]lp(S',M,)-p(D ~, M,)I. Using the triangle inequality,

we obtain

[ a ~ . ( ~ . . ) - - a . . ' ( ~ . ) [ < [2p(S', m . ) + o (s ~, D ~) ] P (S', D ' ) < (2O (S', M,) +~) ~. We put e~,=p(S*, M,); then for ~ ,

la..' (B..)-a..(B..)l<~3p(S', m.)e. 2. Let

p(S~,M,)=O.

In this case

la..'(B.)-a.(B..)l=

p(D~,M,)<-p(D',Su)<~e

I-, I

I-<

p'(D-,m,) i+pt(D', M.) ~

(2) and consequently

P(D"'M') g. i+pt(D-,M.)

Clearly, o(D~,M.)

't +pt (D ~, M,)

~<_i

2

for any

D ~ and

M,.

Then

la.. '(H.=)-a.,(B.~) [<~/2. We ass LL~e

(3)

137

+00 f o r p(S~,M,)=O, e"(J~u') ---~ t~., f o r p (Su, ~1,) >.0, I i for i~=u, ]4=v, ~,,(B~,)= 3p(Su,M.) for i = a , j = v '/z for i=a, ]=o

and and

P(S~,M~) >0,

p(Su,1~[,)=O.

In view of Eqs.(2) and (3), the operators ~"", u=l, 2,...,q, v=l, 2, .... l are completely stable. But then it directly follows from the lemma that the operator B'(~,S r is also completely stable since any operator L(B) can be expressed by a linear combinstion of operators ~=~. Now, since the recognition algorithm A" is correct, we have g~(A')#A for all i and ], and therefore the conditions of Theorem 3 are satisfied. Hence A'is stable. The theorem has been proved. The authors are grateful to Yu.I. Zhuravlev for his interest. REFEP~NCES i. ZHURAVLEV Yu.I., The correct alqebras for incorrect (heuristic) algorithms.I Kibernetika 4, 14-21, and II 6,21-27, 1977. 2. KRASNOPROSHIN V.V. and LEPESHINSKII N.A., On the effectiveness of pattern recognition. Vestisi AN BSSR. Ser. fiz.-mat, navuk, 6, 32,36, 1977. 3. KASHKEVICH S.I. and KRASNOPROSHIN V.V., Recognition algorithms with generalized standards. In: Tezisy dokl. I Grodnenskoi knonf, molodykh uchenykh. Chast' III, 88-89, Grodno, 1979.

Translated by W.C.