- Email: [email protected]
The construction of an optimal linear threshold decision rule
A. I. Zelichenko
188
8. IVANOV, M. Ya., KRAIKO, A. N. and MIKHAILOV, N. V. A method of through computation threedimensional supersonic flows. I. Zh. @hid. Mar. mar. Fir., 12,2,441-463,1972.
for two-and
9. IVANOV, M. Ya. and KRAIKO, A. N. A method of through computation for two-and threedimensional supersonic flows. II. Zh. vjGhisl.Mat. mat. Fiz., 12,3,805-813, 1972. 10. GODUNOV, S. K., ZABRODIN, A. V. and PROKOPOV, G. P. A computational scheme for twodimensional non-stationary problems of gas-dynamics and calculation of the flow from a shock wave approaching a stationary state. Zh. vjdd. Mut. met. Fiz., 1,6, 1020-1050, 1961.
U.S.S.R. Comput. Maths. Math. Phys. Vol. 20, No. 3, pp. 188-199, Printed in Great Britain
1980 0041~5553/80/030188-12$07.50/O Q 1981. Pergamon Press Ltd.
THE CONSTRUtXION OF AN OF’TIMAL LINEAR THRESHOLD DECISION RULE*
A.I. ZELICHENKO Moscow (Received 11 March 1979; revised 28 Januaty 1980)
AN ALGORITHM is proposed for solving the problem of constructing an optimal linear threshold decision rule, when the quality functional is a fraction of the correct predictions. In [ 1] it was shown that every recognition algorithm A can be represented as the successive satisfaction of the algorithms B and C. The algorithm B (the recognition operator) constructs from the initial information Z about the objects S1, . . . , S4 the matrix of real numbers {%} YX~, whose elements are interpreted as values of the proximity of the i-th object to the j-th class. The algorithm C (decision rule) given the matrix {aij},x, constructs the matrix {b
zk-1 *Zh. vj%hisl.Mat. mat. Fiz., 20,3,724-736,
bfaa+b,:,=&
1980.
(1)
Optimal linear threshold decision rule
189
if the i-th object belongs to the i_class, and the inequality 1
c bLa,k+br:~Cc~j’,
(2)
R-l
if the i-th object does not belong to the i-th class. By hypothesis j-1,2,.
Cij’
. . ,1.
(3)
The problem of constructing the optimal linear threshold decision rule reduces to the construction (and decision) of the maximal consistent subsystem of the system obtained by the union of all the inequalities (1) and (2), such that among its decisions there exist decisions satisfying conditions (3). There exists an extensive class of problems of discrete optimization which can be solved by a unified method - by the method of reduction to the socalled problem of fmding the upper zero with maximal norm (or simply, the maximal upper zero) of some Boolean function f(o) from the subclass of monotonic Boolean functions: ~^EK," Timplies that f(Z) = 1. The norm of a Boolean vector is defined as the number of its unit coordinates.] [A step of the algorithm is defined as the determination of the value of f(;;;> in our case, the verification of the consistency of some system of linear inequalities).] This class of problems may be described as follows. Let there exist a set N consisting of n objects (of any kind whatever), and let $’ be some one-place predicate, which it is meaningful to consider on any subset N’ s N, the predicate $F possessing the following property (generalized monotonicity): %!(M, N)c!W(M)&%!(M,
M’)=P(M’),
where A (M, N) is the two-place predicate “M is contained in N (in the set-theoretic sense)“. If the optimization problem consists of finding the maximal (in power) subset No 2 N such that 9 (No) = 0, then this problem can be reduced to the problem of finding the maximal upper zero of the monotonic function: fp(or,.
. . ,an)=
1, 0
if 9 (Na) = 1, otherwise
where N Z consists of the il -th , . . . , ik-th elements of N (here and below, wherever nothing is said to the contrary, it is assumed that il , . . . , ik are the numbers of unit coordinates in Z). Thereby a subclass K~P” is extracted such that for all the functions of K ,.=, and only for them, there exist discrete optimization problems (that is, pairs (9, N) ) , reducible to a search for the maximal upper zero of these functions in the sense indicated above. The class K pn may either be identical with Mn or be strictly contained in Mn .
190
A. I. Zelichenko
It is easy to see that a large number of discrete optimization problems arising in applications reduce to the problem of finding the maximal upper zero. Examples of such problems can be found, in particular, in [4,5] (for the case KPn==M”) and in [6,7] (for the case Kp”cM”) ; in this paper some problems are also considered where K,p”cM”. In [2] it is shown (and this easily follows from the description given above), that the problem of extracting the maximal consistent subsystem from a system of linear inequalities (to which the problem of constructing the optimal linear threshold decision rule reduces), can in its turn be reduced to the problem of fmding the maximal upper zero of some monotonic Boolean function fAE Kqineq C Mq (see [7] for the definition Of K’lineq and some of its properties). The problem of finding the maximal upper zero for the case K~“==M” was solved in [3], where an algorithm was presented which terminates its work in not more than Ciqi2 1+ 1 steps. It can be shown that the additional constraint f E Kqhq the work of the algorithm of [3].
does not lead to an acceleration of
To verify this it is necessary to show, for example, that in Kqineg, firstly, there exists a function fao,with maximal upper zero ipb in the q/Zth layer (the tth layer is the set of Boolean collections with norm i; for convenience we will consider 4 to be even), and secondly, for any collection ;;;1 of the (4/2 + l)-th layer there exists a monotonic function fzlsuch that r;,(&) -0 and j,;,(%) -0, and for all other collections iii’ of the (q/2 + I)-th layer f,-,&‘) -1. In zO let the units be situated in the il -th, . . . ,612 -th places, then fzoisa monotonic function with the set of upper zeros of the two collections ZO = (ii;lO. . . Zqo) and ZO-= @lo. ..Zqo); faoEKa ~eq,since it is equivalent to the system of linear inequalities whose il -th, . . . , iqZ2-th inequalities are x < 0, the remaining inequalities are x > 1. In ;;;1 let the units be situated at the a.ndbyJtheset ci~,..., i& - * * Jq/2+l -th places. We will denote by Z the set {ii, . . . , i,lz} lq/2+1). We note that 1Zn JI = p > 1. We immediately indicate a system of linear inequalities which is equivalent to the function fzl,mentioned above. For I= (I,&. . . , q}
the i-thinequality is as follows:
xto,
if
C-1,
if
ez\{znr>, i=wnr>,
2-q
if
iEZf-/l,
x=-3,
if
i= (1, 2,. . . , q}\{ZlJJ).
If we consider that the system of linear inequalities from which, in order to construct the optimal decision rule it is necessary to extract the maximal consistent subsystem, consists of ql inequalities, then there exists the possibility that the construction of the optimal decision rule wilI require c&‘Z~~+’ steps, each step involving a fairly laborious procedure of checking the consistency of some system of linear inequalities. In this paper the problem of constructing the optimal linear threshold decision rule is reduced to the problem of finding the maximal upper zero of a monotonic function from another subclass of monotonic Boolean functions K&,, - We present an algorithm for fmding the maximal upper zero of a monotonic function of this subclass which ends its work in Z(q- l+ 1) Cq’steps (in the case q > I of practical interest for recognition problems - after less than Z(q- I+ 1) q1 steps), ZCqZ of the
Optimal linear threshold decision rule
191
steps being solutions of systems of I linear equations, and l(q- r) Cqzsteps involving the calculation of the scalar product of two (Z+1)dimensional Euclidean vectors (each step is one scalar product). In section 1 the problem of constructing a linear threshold decision rule is reduced to the solution of 1 problems of finding the maximal upper zero of the monotonic function f(q with the constraint f (a) EKqYHP,where G.i, is some subclass of the class of monotonic Boolean functions of 4 variables MS, which arises when solving the problem of fmding the maximal subsystem of a consistent system of linear inequalities with an unbounded polyhedron of solutions. In section 2 a property of functions of Ki,,. important for solving the problem of finding the maximal upper zero, is proved. In section 3 an algorithm for solving this problem is presented which ends its work in not more than (q-l+1)C9z steps.
§1 We first transform inequalities (1) and (2) to the form 1
bkfafr>Czj.
(41)
where c2,=c2,‘-b,:,, System (4) obtained by combining alI the inequalities (41) Ct,=Clj ‘4,. and (4& can be subdivided into I subsystems, each of them containing its own group of Z+2 independent variables b,P (i=l, 2, . . . , Z), clp, czp (hereandbelowp=1,2,...,L For this subdivision we include in the pth subsystem the 4 inequalities 1 bkPaik > czp, if the i-th object belongs to the pth class,
c
k=l
1
I: brPaik <
clp, if the i-th object does not belong to the pth class.
kc1
It is obvious that the maximal consistent subsystem of system (4) subject to condition (3) is the union of the maximal consistent subsystems of (5,). Therefore, the problem of finding the maximal consistent subsystem of the system (4) subject to condition (3) reduces to the solution of the problems of fmding the maximal consistent subsystem of system (5p) subject to condition (3). We note that condition (3) breaks up into 1 conditions tip-2P.
(3p)
It is obvious that only condition (3p) affects the system (5,), since the remaining conditions do not impose constraints on the variables of the system (5,). We now show that in the absence of condition (3p) system (5,) is consistent. Indeed, let us substitute in the left sides of the inequalities of system (5,) any vector bp E RI (an I-dimensional Euclidean space). We obtain on the left sides, respectively, the values
d,(P;.. . , dJP’. We introduce the sets Qr (p) and Q2(P), consisting of the numbers of the objects belonging to and not belonging to the p-th class, respectively. We choose clp and czp as follows:
A. I. Zelichenko
192
cl,
>
max dIP’,
czP c
is@(P)
min dip’. fOQl(P)
It is obvious that in the absence of condition (3J the vector ( (bP) ‘, clp, CIP) ' is a solution of system (SP). Therefore, the problem of finding the maximum consistent subsystem of the system (SP) subject to condition (3P) reduces to the following. Let there exist the system
aqbxb’c2 b-i
and the inequality Ci
(7)
It is required to fmd the maximum subsystem of system (6) consistent with inequality (7), it being assumed that (6) is consistent. This problem is equivalent to the problem of finding the maximum upper zero of the monotonic Boolean function defined as follows: 1, f (al, . . . , aJ = 0
if the subsystem of the i,-th, . . . , ik-th inequalities of (6) is inconsistent with (7),
(8)
otherwise.
The monotonicity of (8) is obvious. It is easy to show that it is not for any monotonic function that a system (6) equivalent to it and an inequality (7) exist. Therefore, there naturally arises a subclass 1Ip of monotonic Boolean functions for which system (6) and an inequality (7) equivalent to it exist in the sense indicated above. We introduce the following system of inequalities:
OptimaI linear threshold decision rule
193
1
c
alkxc,
A-l .
.
.
.
0
.
(9)
1
c
ar+dPc2,
k-1 .
.
.
.
*
.
1
c
aqhxk>c2
k-i
(DO is some constant). System (9) is obtained by combining the system (6) with the equation
(10)
We will now prove the necessary and sufficient conditions for subsystem (6) to be consistent with inequality (7). We first introduce two systems of linear inequalities. Systems (6) and (9) can be written in the form A1y < b, and Azy < b respectively (A,, AZ are matrices of dimension qX (Z-i-2)) y= We will be interested in their subsystems consisting of the il -th, (x’, ci, c+R1+*, b,, b+RQ). . . . , ik-th inequalities A;“ycb,“,
(11)
A,“ycb>
(12)
(the upper suffer 01indicates the fact that the corresponding vectors and matrices contain only the i1-th, . . . , ik-th vectors and matrices without the suffm). Theorem I System (11) is consistent with (7) provided that the polyhedron of its solutions does not come into contact with the hyperplane cl = q (see [8] ) when and only when (12) has an unbounded solution. Proo$ Necessity. If (11) is consistent with (7), then (12) has a bounded solution. We note that if (11) is consistent with (7), then points satisfying (11) exist on the hyperplane cl = ~2, since if all the solutions of (11) were to satisfy (7) strictly, then all the solutions of (6) also would strictly satisfy (7) and none would satisfy the inequality cl > ~2, which cannot be the case (see the proof of the consistency of system (5 )). We consider the vector Y = (xl, . . . , x1, cl)*, that is, Y belongs to the hyperplane cl = c2 in RP+2.
194
A. I. Zelichenko
Let Y satisfy (11); then Y is an interior point of the set of solutions of (1 l), that is, a S-neighbourhood of Y exists which belongs to the set of solutions of (11). It is obvious that in this case an &neighbourhood of the point arY, cu>O, also belongs to the set of solutions of (11). We put cr>A/S. In this case the 016neighbourhood of the point CUY contains points belonging to the hyperplane (10) at the same time satisfying (11). Necessity is proved. Sufficiency. If (12) has a bounded solution, then (11) is consistent with (7). We will prove the following sufficient statement: if the non-strict system (11) and the non-strict inequality (7) (that is, system (11) and inequality (7) in which all the signs > and < are replaced by > and G) are consistent only at the point (0, . . . , 0) (and such a consistency of them is equivalent to the inconsistency of the strict (11) and (7) provided that the polyhedron of solutions (11) does not come into contact with the hyperplane cl = cz), then the set of solutions of (12) is bounded. We will give an indirect proof of this statement. In this case vectors Y,, Yd?‘+*, 1Y. j’=A,>A, 1Y 1=I, exist such that Yo+aY, Oazb=J, satisfies(12), and therefore simultaneously satisfies (10) and (11). It is obvious that in this case CYY satisfies the non-strict inequality (7) for -W 0) also do not satisfy thenon-strict system (1 l), and putting cu’>o~>A1 /S, we arrive at the result that Yo + a’Y do not satisfy the non-strict system (11). This contradiction proves the sufficiency of the statement of Theorem 1. Theorem 1 implies a result important in what follows. The problem of constructing an optimal decision rule (accurate to the satisfaction of the condition that&e polyhedron of solutions (11) does not touch the hyperplane cl = c2) is equivalent to the problem of finding the maximum subsystem of system (9) possessing an unbounded poly hedron of solutions. The latter problem, like the preceding one, reduces to the problem of fmding the maximum upper zero of the monotonic Boolean function f” with the additional constraint
We define f* as follows:
II, f’ (al . . . aq)
F
if the subsystem (12) has a bounded polyhedron of solutions,
1’ 0
(13)
otherwise.
The monotonicity of (13) is obvious. It is easy to show that a system (9) corresponding to it in the sense of (13) does not exist for any monotonic Boolean function. Therefore, it is natural to of monotonic Boolean functions for which there exists a system (9) introduce the subclass &$,, corresponding to them in the sense of (13). Below we will be interested in the number of steps of the algorithm necessary to find the maximum upper zero of the monotonic function f* given the a prior information P&rP* (Incidentally, we note that the algorithm for fmding the maximum upper zero off‘ proposed in this paper can also be used to decipher it without any changes.)
Optimal linear threshold decision rule
195
In this section we will be interested in such properties of functions of K;,, , important for algorithms for finding the maximum upper zero, as the upper bound of the number of its upper zeros. It wih be proved that the number of the upper zeros of the functionfc(Z) cannot exceed 2C4’/(1+1). We will prove this for a more general case. Let Axcb, where
A- {cU}qX(l+i), &R*,
x&‘+’
(14)
is a system of linear inequalities.
We introduce the monotonic Boolean function
f’(a)=p(a,...a,) =
1, if 0
(15)
A6x < ba has a bounded solution, otherwise.
Theorem 2 The function f+(Z) has not more than 2C4’/(1+1) upper zeros. Proof: We note the one-to-one correspondence between the upper zeros off + and those unbounded linear polyhedra in R z+l which are the solutions of the subsystem of system (14) such that the addition to this subsystem of one more inequality from (14) makes its solution bounded. Below, for brevity, we wih call such polyhedra upper-zero polyhedra and the subsystems corresponding to them upper-zero systems. We now introduce the concept of an Lsystem, fundamental for what follows. (For the present we will provisionally regard all the inequalities (14) as non-strict. This makes it convenient when using the terminology of [8]. If the strictness of (14) is essential, this case will be described separately.) Definition. The Lsystem of an upper-zero system will be defined as a subsystem of rank 1 of it, consisting of 1 inequalities with I+1 unknowns, which is limiting for the upper-zero system (here and below the terminology and results of Chap. 1 of [8] are used) and for which the following condition is satisfied. Condition. For any p> 0 there exists x E R !+l- the nodal solution of the Lsystem satisfying the upper-zero system, such that Ixl>p. Lemma 1. Lsystems exist in the upper-zero system and there are not less than 1+1 of them. Roof: The solution of the upper-zero system is an unbounded polyhedron. We note (this is easily obtained from the discussion of Theorem 1.15 in [8]), that the rank of the upper-zero system equals Z+l. From the first property of the faces of the polyhedron (see [8] , chap. 1, section 3) it follows that an upper-zero polyhedron contains faces of all dimensions from 0 to 1. Considering property 3 (there dsoj,it is easy to obtain that the unboundedness of the upper-zero polyhedron implies, in particular, the existence of onedimensional unbounded faces, since otherwise all the
A. I. Zelichenko
196
2dimensional faces are bounded, and then also all the 3dhnensional faces are bounded, and so on; finally we obtain the boundedness of the upper-zero polyhedron itself. Every one-dimensional face corresponds to a limiting subsystem of 1 inequalities of rank I of the upper-zero system. We prove the second statement of the lemma. We add to the upper-zero system one more inequality from (14): f(x) =G0, f( x) is a linear function of x. The polyhedron of the resulting system will be bounded, by the definition of an upper-zero system. We consider that l-dimensional (bounded) face of it which corresponds to the I-dimensional hyperplane f(x) = 0. It is an obvious fact that the number of Lsystems is not less that the number of vertices (O-dimensional faces) of the face considered, which in its turn is a polyhedron. But from the discussion of Theorem 1.15 of [8] we easily obtain the following statement proving the second part of Lemma 1: in an Edimensional space a bounded polyhedron has not less that 1+1 vertices. The proof of Lemma 1 is complete. Lemma 2
One subsystem cannot be an Lsystem for more than two upper-zero systems. Proof: Let II be a subsystem of the system (14) of 2 inequalities with Z+l unknowns of rank 1. The set of its nodal solutions can be represented in the form Y (t) =x,+tx,
t=(--co,
+a).
Let
A-x>b-
(16)
be the system of inequalities of the system (14) remaining (after excluding 4[) and Ar’ be the i-th row of the matrix A’. Each inequality of (16) can be referred to one of the following four classes. chs
I.
(A,-, x) XI.
Class II.
(Ai-, x) (0.
class III.
(A,-, x) =O, (Ai-, xo) >bbi.
ass IV.
(Ai-, x) -0,
(A<-, x,) -=S.
We will show that the upper-zero system for which Q is an JI -system may be one of two: besides II it contains all the inequalities either of the classes I and III, or of the classes II and III. The systems obtained in these two cases w-illbe denoted by B1 and V. respectively. We will show that either +?I[’and/or 3’ is the upper-zero system, or there does not exist at all an upper-zero JI -system. Indeed, 2l’ and 11’ correspond to unbounded polysystem for which II is the hedra. The unboundedness of these polyhedra follows from the fact that the positive, t > 0 (respectively, negative, t < 0), part of the straight line Y(t) satisfies 2P (respectively, P) . The strict positiveness (negativeness) of (Ai-, x) implies the existence of an e-neighbourhood of the nodal solutions of g, satisfying +2l’ (respectively, VP). We now consider the two cases: 1) on the addition of any inequality from (16), not occurring in 91l (respectively, II*), the polyhedron of ‘%’ (respectively, a*) becomes bounded; 2) in (16) there exists an inequality, not ocurring in ?l’ (respectively, gZ), on whose addition to the polyhedron of 91’ (respectively,4 V) it remains bounded.
Optimal linear threshold decision rule
197
Case 1) obviously means that a1 (respectively, m2) is an upper-zero system and 2l is the subsystem of it which is the Lsystem for it. For case 2) we show that there does not exist an upper-zero system for which 8f is the Lsystem. Let us assume the opposite, that is, that there exists an upper-zero system including besides the inequalities of the classes I (respectively, II) and III also inequalities of the classes II (respectively, I) and/or IV, where % occurs in -$l as a subsystem and % is the Lsystem for @. To show that this is impossible we consider two cases. Case 1. In % inequalities of the class IV occur. In this case Y(t) does not satisfy these inequalities (and each of them separately), since for the inequalities of the class IV, by the definition of this class, we have (b-, xo)b’ of the classes r, r~ { 1, 2). The following relations hold:
(Al, Y(t)
)=
(A’, x,) +t (A’x) >b’*t
(A2,Y(t))=(A2,xcJ+t(A2x)>b2=+t<
>
(A’, x,) (A’, x)
,
b2- (A2, x,) (A2,X)
,
b’-
that is, the set of possible values oft is bounded (or empty), which is impossible by the definition of an Lsystem. Therefore, it has been shown that every subsystem of the system (14) of 2 inequalities cannot be an Lsystem for more than two upper-zero systems (and it has been shown how to construct them). The proof of Lemma 2 is complete. Now, recalling the result of Lemma 1, to obtain the statement of Theorem 2 it is sufficient to note that there cannot be more than Cqrdifferent Lsystems. This fact is quite obvious. The proof of Theorem 2 is complete. From Theorem 2 it immediately follows that the montonic function (13) also has not more than 2C41/(Z+1)upper zeros. We will prove another result necessary below to justify the algorithm. Theorem 3 Every upper-zero system 9 can be represented in the form S1 (respectively Lsystem ‘Sl of it with the set of nodal solutions Y(t) = x. t tx.
a2) for some
Proof: We consider three cases: 1) the upper-zero system !B ’ is satistied by the whole set Y(t), 2) b is satisfied by the part of the set Y(t), t>a+, 3) 9 is satisfied by the part of the set Y(t), f
A. I. Zelichenko
198
93
We now pass to the description of the algorithm which completes in I(@+l)C,’ steps the construction of the optimal decision rule. We first describe the algorithm for finding the maximum upper zero. Algorithm. 0.
k-0 , x=0.
1.
i:,-i+1,
2. Ifii;=(l,...,
a:+(i),
ccr,=.
. .=afl=l.
l), then end the work, otherwise pass to part 3.
3. Include in 91 the (il, . . . , il)-th inequalities of the system (9) and by means of the construction used in the proof of Lemma 2, form P[’ and 91*. 4. If Z+max (ISr’l,
IVl),
5. n:=Z+max (l%‘l, pass to part 1.
ISr”l)> n, then pass to part 5, otherwise pass to part 1. IV]),
i[:=-ql,
ii:=%],
j={l,
2},
The operator 8 (i) generates the i-th collection of length q(i<$‘), i>C,! then @(i)=(l...l).
lSVI=max
(/VI,
containing 2units. If
From the results of Theorems 2 and 3 it is easy to see that the algorithm described constructs the system ihit&J$, which isthe most powerful upper-zero system for (9). It is obvious that the number of steps of this algorithm necessary to end its work will always be Cql. Here by a step we mean the construction of the systems 91’ and Q’. This is connected with the deftition of the direction x and with the (q-l)-fold calculations of the scalar product of two (Z+l)dimensional vectors. Returning to the old definition of step, it can be said that the work of the algorithm will end after Cc+l+l) steps. We will now describe the complete algorithm for constructing the optimal linear threshold decision rule. The algorithm described for finding the maximum upper zero of (15) will be denoted by Z, the matrix of parameters of the decision rule bi’. * . Wm2 . . . . . . . bl’ . . . Vwzl
will be denoted by B, its j-th row by hi(c). Algorithm for the construction of the optimal decision rule 0. j: = 0.
1. j: =j+l. 2. If j < I, then pass to part 3, otherwise end the work of the algorithm.
Optimal linear threshold decision rule
199
3. Form the system (9j) (the subscript j indicates which of the rows of the matrix B is used as the independent variables of the system (9)). Apply to the system obtained the algorithm Z:
Zt (9j) lcB>I) approximately f(r l. Au the steps of the algorithms described can easily be realized on a computer with an entirely admissible computing time. In conclusion we note’the following important factor. In actual pattern recognition problems the coefficients ~j are always determined with some tolerance, that is, we actually always have not the coefficients Urisbut some intervals (qj-, +j+), in which the qj OCCUI: atjE
(Q-7
Q+)
9
i-l,
2,. . . , q,
j=l,
2,. . . , 1.
This fact helps us to avoid troublesome constraints imposed by Theorem 1 (we recall that they consist of forbidding contact of the hyperplane cl = c2 with the solution cone of (11)). Indeed, varying the coefficients +j in the corresponding intervals, it is always possible to arrange that such a contact does not occur. The author is deeply indebted to Yu. I. Zhuravlev for suggesting the problem and for his interest and to Kh. A. Madatyan for valuable comments and editorial assistance. Translated by J. Berry.
REFERENCES 1. ZHURAVLkV, Yu. I. Correct algebras for sets of ill-posed (heuristic) algorithms. I. Kibemetiko, No. 4,14-21, 1977. 2. ZHURAVLh,
Yu. I. Non-parametricpattern recognition problems. Kibemetiku, No. 6,93-103,1976.
3. KATERINOCHKINA, N. N. The search for the maximumupper zero of a monotonic function of the algebra of logic. Dokl. Akad. Nauk SSSR. 224,3,557-560,197s.
4. ZHURAVLh, Yu. I. On algorithms for the extraction of aggregates of essentiaI variables of functions of the algebra of logic not defined everywhere. In: Problems of cybernetics (ProbL kiiemetiki). No. 11, 271-275, “Nauka”, Moscow, 1964. 5. KOROBKOV, V. K. On some numerical problems of linear programming. In: Problems of cybernetics (Probl. kiiemetiki), No. 14,197-299, “Nauka”, Moscow, 1965. 6. ZHURAVLIk, Yu. I. On a class of functions of the algebra of logic not defined everywhere. In: Discrete analysis (Diskretnyi analiz), No. 2,23-27, In-t matem. SO Akad. Nauk SSSR, Novosibirsk, 1964. 7. ZELICHENKO, A. I. On the correspondence of systems of Iinear inequalities to monotonic Boolean functions. Zh. vychisl. Mat. mat. Fiz., 19,6,1543-1554, 1979. 8. CHERNIKOV, S. N. Linear inequalities (lineinye neravenstva), “Nauka”, Moscow, 1968.
188
8. IVANOV, M. Ya., KRAIKO, A. N. and MIKHAILOV, N. V. A method of through computation threedimensional supersonic flows. I. Zh. @hid. Mar. mar. Fir., 12,2,441-463,1972.
for two-and
9. IVANOV, M. Ya. and KRAIKO, A. N. A method of through computation for two-and threedimensional supersonic flows. II. Zh. vjGhisl.Mat. mat. Fiz., 12,3,805-813, 1972. 10. GODUNOV, S. K., ZABRODIN, A. V. and PROKOPOV, G. P. A computational scheme for twodimensional non-stationary problems of gas-dynamics and calculation of the flow from a shock wave approaching a stationary state. Zh. vjdd. Mut. met. Fiz., 1,6, 1020-1050, 1961.
U.S.S.R. Comput. Maths. Math. Phys. Vol. 20, No. 3, pp. 188-199, Printed in Great Britain
1980 0041~5553/80/030188-12$07.50/O Q 1981. Pergamon Press Ltd.
THE CONSTRUtXION OF AN OF’TIMAL LINEAR THRESHOLD DECISION RULE*
A.I. ZELICHENKO Moscow (Received 11 March 1979; revised 28 Januaty 1980)
AN ALGORITHM is proposed for solving the problem of constructing an optimal linear threshold decision rule, when the quality functional is a fraction of the correct predictions. In [ 1] it was shown that every recognition algorithm A can be represented as the successive satisfaction of the algorithms B and C. The algorithm B (the recognition operator) constructs from the initial information Z about the objects S1, . . . , S4 the matrix of real numbers {%} YX~, whose elements are interpreted as values of the proximity of the i-th object to the j-th class. The algorithm C (decision rule) given the matrix {aij},x, constructs the matrix {b
zk-1 *Zh. vj%hisl.Mat. mat. Fiz., 20,3,724-736,
bfaa+b,:,=&
1980.
(1)
Optimal linear threshold decision rule
189
if the i-th object belongs to the i_class, and the inequality 1
c bLa,k+br:~Cc~j’,
(2)
R-l
if the i-th object does not belong to the i-th class. By hypothesis j-1,2,.
Cij’
. . ,1.
(3)
The problem of constructing the optimal linear threshold decision rule reduces to the construction (and decision) of the maximal consistent subsystem of the system obtained by the union of all the inequalities (1) and (2), such that among its decisions there exist decisions satisfying conditions (3). There exists an extensive class of problems of discrete optimization which can be solved by a unified method - by the method of reduction to the socalled problem of fmding the upper zero with maximal norm (or simply, the maximal upper zero) of some Boolean function f(o) from the subclass of monotonic Boolean functions: ~^EK,"
M’)=P(M’),
where A (M, N) is the two-place predicate “M is contained in N (in the set-theoretic sense)“. If the optimization problem consists of finding the maximal (in power) subset No 2 N such that 9 (No) = 0, then this problem can be reduced to the problem of finding the maximal upper zero of the monotonic function: fp(or,.
. . ,an)=
1, 0
if 9 (Na) = 1, otherwise
where N Z consists of the il -th , . . . , ik-th elements of N (here and below, wherever nothing is said to the contrary, it is assumed that il , . . . , ik are the numbers of unit coordinates in Z). Thereby a subclass K~P” is extracted such that for all the functions of K ,.=, and only for them, there exist discrete optimization problems (that is, pairs (9, N) ) , reducible to a search for the maximal upper zero of these functions in the sense indicated above. The class K pn may either be identical with Mn or be strictly contained in Mn .
190
A. I. Zelichenko
It is easy to see that a large number of discrete optimization problems arising in applications reduce to the problem of finding the maximal upper zero. Examples of such problems can be found, in particular, in [4,5] (for the case KPn==M”) and in [6,7] (for the case Kp”cM”) ; in this paper some problems are also considered where K,p”cM”. In [2] it is shown (and this easily follows from the description given above), that the problem of extracting the maximal consistent subsystem from a system of linear inequalities (to which the problem of constructing the optimal linear threshold decision rule reduces), can in its turn be reduced to the problem of fmding the maximal upper zero of some monotonic Boolean function fAE Kqineq C Mq (see [7] for the definition Of K’lineq and some of its properties). The problem of finding the maximal upper zero for the case K~“==M” was solved in [3], where an algorithm was presented which terminates its work in not more than Ciqi2 1+ 1 steps. It can be shown that the additional constraint f E Kqhq the work of the algorithm of [3].
does not lead to an acceleration of
To verify this it is necessary to show, for example, that in Kqineg, firstly, there exists a function fao,with maximal upper zero ipb in the q/Zth layer (the tth layer is the set of Boolean collections with norm i; for convenience we will consider 4 to be even), and secondly, for any collection ;;;1 of the (4/2 + l)-th layer there exists a monotonic function fzlsuch that r;,(&) -0 and j,;,(%) -0, and for all other collections iii’ of the (q/2 + I)-th layer f,-,&‘) -1. In zO let the units be situated in the il -th, . . . ,612 -th places, then fzoisa monotonic function with the set of upper zeros of the two collections ZO = (ii;lO. . . Zqo) and ZO-= @lo. ..Zqo); faoEKa ~eq,since it is equivalent to the system of linear inequalities whose il -th, . . . , iqZ2-th inequalities are x < 0, the remaining inequalities are x > 1. In ;;;1 let the units be situated at the a.ndbyJtheset ci~,..., i& - * * Jq/2+l -th places. We will denote by Z the set {ii, . . . , i,lz} lq/2+1). We note that 1Zn JI = p > 1. We immediately indicate a system of linear inequalities which is equivalent to the function fzl,mentioned above. For I= (I,&. . . , q}
the i-thinequality is as follows:
xto,
if
C-1,
if
ez\{znr>, i=wnr>,
2-q
if
iEZf-/l,
x=-3,
if
i= (1, 2,. . . , q}\{ZlJJ).
If we consider that the system of linear inequalities from which, in order to construct the optimal decision rule it is necessary to extract the maximal consistent subsystem, consists of ql inequalities, then there exists the possibility that the construction of the optimal decision rule wilI require c&‘Z~~+’ steps, each step involving a fairly laborious procedure of checking the consistency of some system of linear inequalities. In this paper the problem of constructing the optimal linear threshold decision rule is reduced to the problem of finding the maximal upper zero of a monotonic function from another subclass of monotonic Boolean functions K&,, - We present an algorithm for fmding the maximal upper zero of a monotonic function of this subclass which ends its work in Z(q- l+ 1) Cq’steps (in the case q > I of practical interest for recognition problems - after less than Z(q- I+ 1) q1 steps), ZCqZ of the
Optimal linear threshold decision rule
191
steps being solutions of systems of I linear equations, and l(q- r) Cqzsteps involving the calculation of the scalar product of two (Z+1)dimensional Euclidean vectors (each step is one scalar product). In section 1 the problem of constructing a linear threshold decision rule is reduced to the solution of 1 problems of finding the maximal upper zero of the monotonic function f(q with the constraint f (a) EKqYHP,where G.i, is some subclass of the class of monotonic Boolean functions of 4 variables MS, which arises when solving the problem of fmding the maximal subsystem of a consistent system of linear inequalities with an unbounded polyhedron of solutions. In section 2 a property of functions of Ki,,. important for solving the problem of finding the maximal upper zero, is proved. In section 3 an algorithm for solving this problem is presented which ends its work in not more than (q-l+1)C9z steps.
§1 We first transform inequalities (1) and (2) to the form 1
bkfafr>Czj.
(41)
where c2,=c2,‘-b,:,, System (4) obtained by combining alI the inequalities (41) Ct,=Clj ‘4,. and (4& can be subdivided into I subsystems, each of them containing its own group of Z+2 independent variables b,P (i=l, 2, . . . , Z), clp, czp (hereandbelowp=1,2,...,L For this subdivision we include in the pth subsystem the 4 inequalities 1 bkPaik > czp, if the i-th object belongs to the pth class,
c
k=l
1
I: brPaik <
clp, if the i-th object does not belong to the pth class.
kc1
It is obvious that the maximal consistent subsystem of system (4) subject to condition (3) is the union of the maximal consistent subsystems of (5,). Therefore, the problem of finding the maximal consistent subsystem of the system (4) subject to condition (3) reduces to the solution of the problems of fmding the maximal consistent subsystem of system (5p) subject to condition (3). We note that condition (3) breaks up into 1 conditions tip-2P.
(3p)
It is obvious that only condition (3p) affects the system (5,), since the remaining conditions do not impose constraints on the variables of the system (5,). We now show that in the absence of condition (3p) system (5,) is consistent. Indeed, let us substitute in the left sides of the inequalities of system (5,) any vector bp E RI (an I-dimensional Euclidean space). We obtain on the left sides, respectively, the values
d,(P;.. . , dJP’. We introduce the sets Qr (p) and Q2(P), consisting of the numbers of the objects belonging to and not belonging to the p-th class, respectively. We choose clp and czp as follows:
A. I. Zelichenko
192
cl,
>
max dIP’,
czP c
is@(P)
min dip’. fOQl(P)
It is obvious that in the absence of condition (3J the vector ( (bP) ‘, clp, CIP) ' is a solution of system (SP). Therefore, the problem of finding the maximum consistent subsystem of the system (SP) subject to condition (3P) reduces to the following. Let there exist the system
aqbxb’c2 b-i
and the inequality Ci
(7)
It is required to fmd the maximum subsystem of system (6) consistent with inequality (7), it being assumed that (6) is consistent. This problem is equivalent to the problem of finding the maximum upper zero of the monotonic Boolean function defined as follows: 1, f (al, . . . , aJ = 0
if the subsystem of the i,-th, . . . , ik-th inequalities of (6) is inconsistent with (7),
(8)
otherwise.
The monotonicity of (8) is obvious. It is easy to show that it is not for any monotonic function that a system (6) equivalent to it and an inequality (7) exist. Therefore, there naturally arises a subclass 1Ip of monotonic Boolean functions for which system (6) and an inequality (7) equivalent to it exist in the sense indicated above. We introduce the following system of inequalities:
OptimaI linear threshold decision rule
193
1
c
alkxc,
A-l .
.
.
.
0
.
(9)
1
c
ar+dPc2,
k-1 .
.
.
.
*
.
1
c
aqhxk>c2
k-i
(DO is some constant). System (9) is obtained by combining the system (6) with the equation
(10)
We will now prove the necessary and sufficient conditions for subsystem (6) to be consistent with inequality (7). We first introduce two systems of linear inequalities. Systems (6) and (9) can be written in the form A1y < b, and Azy < b respectively (A,, AZ are matrices of dimension qX (Z-i-2)) y= We will be interested in their subsystems consisting of the il -th, (x’, ci, c+R1+*, b,, b+RQ). . . . , ik-th inequalities A;“ycb,“,
(11)
A,“ycb>
(12)
(the upper suffer 01indicates the fact that the corresponding vectors and matrices contain only the i1-th, . . . , ik-th vectors and matrices without the suffm). Theorem I System (11) is consistent with (7) provided that the polyhedron of its solutions does not come into contact with the hyperplane cl = q (see [8] ) when and only when (12) has an unbounded solution. Proo$ Necessity. If (11) is consistent with (7), then (12) has a bounded solution. We note that if (11) is consistent with (7), then points satisfying (11) exist on the hyperplane cl = ~2, since if all the solutions of (11) were to satisfy (7) strictly, then all the solutions of (6) also would strictly satisfy (7) and none would satisfy the inequality cl > ~2, which cannot be the case (see the proof of the consistency of system (5 )). We consider the vector Y = (xl, . . . , x1, cl)*, that is, Y belongs to the hyperplane cl = c2 in RP+2.
194
A. I. Zelichenko
Let Y satisfy (11); then Y is an interior point of the set of solutions of (1 l), that is, a S-neighbourhood of Y exists which belongs to the set of solutions of (11). It is obvious that in this case an &neighbourhood of the point arY, cu>O, also belongs to the set of solutions of (11). We put cr>A/S. In this case the 016neighbourhood of the point CUY contains points belonging to the hyperplane (10) at the same time satisfying (11). Necessity is proved. Sufficiency. If (12) has a bounded solution, then (11) is consistent with (7). We will prove the following sufficient statement: if the non-strict system (11) and the non-strict inequality (7) (that is, system (11) and inequality (7) in which all the signs > and < are replaced by > and G) are consistent only at the point (0, . . . , 0) (and such a consistency of them is equivalent to the inconsistency of the strict (11) and (7) provided that the polyhedron of solutions (11) does not come into contact with the hyperplane cl = cz), then the set of solutions of (12) is bounded. We will give an indirect proof of this statement. In this case vectors Y,, Yd?‘+*, 1Y. j’=A,>A, 1Y 1=I, exist such that Yo+aY, Oazb=J, satisfies(12), and therefore simultaneously satisfies (10) and (11). It is obvious that in this case CYY satisfies the non-strict inequality (7) for -W
We define f* as follows:
II, f’ (al . . . aq)
F
if the subsystem (12) has a bounded polyhedron of solutions,
1’ 0
(13)
otherwise.
The monotonicity of (13) is obvious. It is easy to show that a system (9) corresponding to it in the sense of (13) does not exist for any monotonic Boolean function. Therefore, it is natural to of monotonic Boolean functions for which there exists a system (9) introduce the subclass &$,, corresponding to them in the sense of (13). Below we will be interested in the number of steps of the algorithm necessary to find the maximum upper zero of the monotonic function f* given the a prior information P&rP* (Incidentally, we note that the algorithm for fmding the maximum upper zero off‘ proposed in this paper can also be used to decipher it without any changes.)
Optimal linear threshold decision rule
195
In this section we will be interested in such properties of functions of K;,, , important for algorithms for finding the maximum upper zero, as the upper bound of the number of its upper zeros. It wih be proved that the number of the upper zeros of the functionfc(Z) cannot exceed 2C4’/(1+1). We will prove this for a more general case. Let Axcb, where
A- {cU}qX(l+i), &R*,
x&‘+’
(14)
is a system of linear inequalities.
We introduce the monotonic Boolean function
f’(a)=p(a,...a,) =
1, if 0
(15)
A6x < ba has a bounded solution, otherwise.
Theorem 2 The function f+(Z) has not more than 2C4’/(1+1) upper zeros. Proof: We note the one-to-one correspondence between the upper zeros off + and those unbounded linear polyhedra in R z+l which are the solutions of the subsystem of system (14) such that the addition to this subsystem of one more inequality from (14) makes its solution bounded. Below, for brevity, we wih call such polyhedra upper-zero polyhedra and the subsystems corresponding to them upper-zero systems. We now introduce the concept of an Lsystem, fundamental for what follows. (For the present we will provisionally regard all the inequalities (14) as non-strict. This makes it convenient when using the terminology of [8]. If the strictness of (14) is essential, this case will be described separately.) Definition. The Lsystem of an upper-zero system will be defined as a subsystem of rank 1 of it, consisting of 1 inequalities with I+1 unknowns, which is limiting for the upper-zero system (here and below the terminology and results of Chap. 1 of [8] are used) and for which the following condition is satisfied. Condition. For any p> 0 there exists x E R !+l- the nodal solution of the Lsystem satisfying the upper-zero system, such that Ixl>p. Lemma 1. Lsystems exist in the upper-zero system and there are not less than 1+1 of them. Roof: The solution of the upper-zero system is an unbounded polyhedron. We note (this is easily obtained from the discussion of Theorem 1.15 in [8]), that the rank of the upper-zero system equals Z+l. From the first property of the faces of the polyhedron (see [8] , chap. 1, section 3) it follows that an upper-zero polyhedron contains faces of all dimensions from 0 to 1. Considering property 3 (there dsoj,it is easy to obtain that the unboundedness of the upper-zero polyhedron implies, in particular, the existence of onedimensional unbounded faces, since otherwise all the
A. I. Zelichenko
196
2dimensional faces are bounded, and then also all the 3dhnensional faces are bounded, and so on; finally we obtain the boundedness of the upper-zero polyhedron itself. Every one-dimensional face corresponds to a limiting subsystem of 1 inequalities of rank I of the upper-zero system. We prove the second statement of the lemma. We add to the upper-zero system one more inequality from (14): f(x) =G0, f( x) is a linear function of x. The polyhedron of the resulting system will be bounded, by the definition of an upper-zero system. We consider that l-dimensional (bounded) face of it which corresponds to the I-dimensional hyperplane f(x) = 0. It is an obvious fact that the number of Lsystems is not less that the number of vertices (O-dimensional faces) of the face considered, which in its turn is a polyhedron. But from the discussion of Theorem 1.15 of [8] we easily obtain the following statement proving the second part of Lemma 1: in an Edimensional space a bounded polyhedron has not less that 1+1 vertices. The proof of Lemma 1 is complete. Lemma 2
One subsystem cannot be an Lsystem for more than two upper-zero systems. Proof: Let II be a subsystem of the system (14) of 2 inequalities with Z+l unknowns of rank 1. The set of its nodal solutions can be represented in the form Y (t) =x,+tx,
t=(--co,
+a).
Let
A-x>b-
(16)
be the system of inequalities of the system (14) remaining (after excluding 4[) and Ar’ be the i-th row of the matrix A’. Each inequality of (16) can be referred to one of the following four classes. chs
I.
(A,-, x) XI.
Class II.
(Ai-, x) (0.
class III.
(A,-, x) =O, (Ai-, xo) >bbi.
ass IV.
(Ai-, x) -0,
(A<-, x,) -=S.
We will show that the upper-zero system for which Q is an JI -system may be one of two: besides II it contains all the inequalities either of the classes I and III, or of the classes II and III. The systems obtained in these two cases w-illbe denoted by B1 and V. respectively. We will show that either +?I[’and/or 3’ is the upper-zero system, or there does not exist at all an upper-zero JI -system. Indeed, 2l’ and 11’ correspond to unbounded polysystem for which II is the hedra. The unboundedness of these polyhedra follows from the fact that the positive, t > 0 (respectively, negative, t < 0), part of the straight line Y(t) satisfies 2P (respectively, P) . The strict positiveness (negativeness) of (Ai-, x) implies the existence of an e-neighbourhood of the nodal solutions of g, satisfying +2l’ (respectively, VP). We now consider the two cases: 1) on the addition of any inequality from (16), not occurring in 91l (respectively, II*), the polyhedron of ‘%’ (respectively, a*) becomes bounded; 2) in (16) there exists an inequality, not ocurring in ?l’ (respectively, gZ), on whose addition to the polyhedron of 91’ (respectively,4 V) it remains bounded.
Optimal linear threshold decision rule
197
Case 1) obviously means that a1 (respectively, m2) is an upper-zero system and 2l is the subsystem of it which is the Lsystem for it. For case 2) we show that there does not exist an upper-zero system for which 8f is the Lsystem. Let us assume the opposite, that is, that there exists an upper-zero system including besides the inequalities of the classes I (respectively, II) and III also inequalities of the classes II (respectively, I) and/or IV, where % occurs in -$l as a subsystem and % is the Lsystem for @. To show that this is impossible we consider two cases. Case 1. In % inequalities of the class IV occur. In this case Y(t) does not satisfy these inequalities (and each of them separately), since for the inequalities of the class IV, by the definition of this class, we have (b-, xo)
(Al, Y(t)
)=
(A’, x,) +t (A’x) >b’*t
(A2,Y(t))=(A2,xcJ+t(A2x)>b2=+t<
>
(A’, x,) (A’, x)
,
b2- (A2, x,) (A2,X)
,
b’-
that is, the set of possible values oft is bounded (or empty), which is impossible by the definition of an Lsystem. Therefore, it has been shown that every subsystem of the system (14) of 2 inequalities cannot be an Lsystem for more than two upper-zero systems (and it has been shown how to construct them). The proof of Lemma 2 is complete. Now, recalling the result of Lemma 1, to obtain the statement of Theorem 2 it is sufficient to note that there cannot be more than Cqrdifferent Lsystems. This fact is quite obvious. The proof of Theorem 2 is complete. From Theorem 2 it immediately follows that the montonic function (13) also has not more than 2C41/(Z+1)upper zeros. We will prove another result necessary below to justify the algorithm. Theorem 3 Every upper-zero system 9 can be represented in the form S1 (respectively Lsystem ‘Sl of it with the set of nodal solutions Y(t) = x. t tx.
a2) for some
Proof: We consider three cases: 1) the upper-zero system !B ’ is satistied by the whole set Y(t), 2) b is satisfied by the part of the set Y(t), t>a+, 3) 9 is satisfied by the part of the set Y(t), f
A. I. Zelichenko
198
93
We now pass to the description of the algorithm which completes in I(@+l)C,’ steps the construction of the optimal decision rule. We first describe the algorithm for finding the maximum upper zero. Algorithm. 0.
k-0 , x=0.
1.
i:,-i+1,
2. Ifii;=(l,...,
a:+(i),
ccr,=.
. .=afl=l.
l), then end the work, otherwise pass to part 3.
3. Include in 91 the (il, . . . , il)-th inequalities of the system (9) and by means of the construction used in the proof of Lemma 2, form P[’ and 91*. 4. If Z+max (ISr’l,
IVl),
5. n:=Z+max (l%‘l, pass to part 1.
ISr”l)> n, then pass to part 5, otherwise pass to part 1. IV]),
i[:=-ql,
ii:=%],
j={l,
2},
The operator 8 (i) generates the i-th collection of length q(i<$‘), i>C,! then @(i)=(l...l).
lSVI=max
(/VI,
containing 2units. If
From the results of Theorems 2 and 3 it is easy to see that the algorithm described constructs the system ihit&J$, which isthe most powerful upper-zero system for (9). It is obvious that the number of steps of this algorithm necessary to end its work will always be Cql. Here by a step we mean the construction of the systems 91’ and Q’. This is connected with the deftition of the direction x and with the (q-l)-fold calculations of the scalar product of two (Z+l)dimensional vectors. Returning to the old definition of step, it can be said that the work of the algorithm will end after Cc+l+l) steps. We will now describe the complete algorithm for constructing the optimal linear threshold decision rule. The algorithm described for finding the maximum upper zero of (15) will be denoted by Z, the matrix of parameters of the decision rule bi’. * . Wm2 . . . . . . . bl’ . . . Vwzl
will be denoted by B, its j-th row by hi(c). Algorithm for the construction of the optimal decision rule 0. j: = 0.
1. j: =j+l. 2. If j < I, then pass to part 3, otherwise end the work of the algorithm.
Optimal linear threshold decision rule
199
3. Form the system (9j) (the subscript j indicates which of the rows of the matrix B is used as the independent variables of the system (9)). Apply to the system obtained the algorithm Z:
Zt (9j) lcB
(Q-7
Q+)
9
i-l,
2,. . . , q,
j=l,
2,. . . , 1.
This fact helps us to avoid troublesome constraints imposed by Theorem 1 (we recall that they consist of forbidding contact of the hyperplane cl = c2 with the solution cone of (11)). Indeed, varying the coefficients +j in the corresponding intervals, it is always possible to arrange that such a contact does not occur. The author is deeply indebted to Yu. I. Zhuravlev for suggesting the problem and for his interest and to Kh. A. Madatyan for valuable comments and editorial assistance. Translated by J. Berry.
REFERENCES 1. ZHURAVLkV, Yu. I. Correct algebras for sets of ill-posed (heuristic) algorithms. I. Kibemetiko, No. 4,14-21, 1977. 2. ZHURAVLh,
Yu. I. Non-parametricpattern recognition problems. Kibemetiku, No. 6,93-103,1976.
3. KATERINOCHKINA, N. N. The search for the maximumupper zero of a monotonic function of the algebra of logic. Dokl. Akad. Nauk SSSR. 224,3,557-560,197s.
4. ZHURAVLh, Yu. I. On algorithms for the extraction of aggregates of essentiaI variables of functions of the algebra of logic not defined everywhere. In: Problems of cybernetics (ProbL kiiemetiki). No. 11, 271-275, “Nauka”, Moscow, 1964. 5. KOROBKOV, V. K. On some numerical problems of linear programming. In: Problems of cybernetics (Probl. kiiemetiki), No. 14,197-299, “Nauka”, Moscow, 1965. 6. ZHURAVLIk, Yu. I. On a class of functions of the algebra of logic not defined everywhere. In: Discrete analysis (Diskretnyi analiz), No. 2,23-27, In-t matem. SO Akad. Nauk SSSR, Novosibirsk, 1964. 7. ZELICHENKO, A. I. On the correspondence of systems of Iinear inequalities to monotonic Boolean functions. Zh. vychisl. Mat. mat. Fiz., 19,6,1543-1554, 1979. 8. CHERNIKOV, S. N. Linear inequalities (lineinye neravenstva), “Nauka”, Moscow, 1968.