- Email: [email protected]
Logic applied to integer programming and integer programming applied to logic
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH ELSEVIER
European Journal of Operational Research 81 (1995) 605-616
Theory and Methodology
Logic applied to integer programming and integer programming applied to logic H.P. Williams Faculty of Mathematical Studies, University of Southampton, Southampton, S09 5NH, UK Received July 1993; revised November 1993
Abstract
The many connections between the methods of Computational Logic and Integer Programming (IP) are surveyed. It is shown how computational problems arising in formal logic can be solved by IP. Also it is shown how the methods of logic are applicable both to modelling and solving IP models. It is shown how Fourier-Motzkin elimination for Linear Programming, when specialised to 0-1 IP models gives rise to the logic method of Resolution. Finally conventional IP methods are applied to solving logical inference problems.
Keywords: Fourier-Motzkin elimination; Logical deduction; Logical resolution; Propositional calculus; Satisfiability; 0-1 integer programming
1. Introduction
There are many cross connections between Logic, Artificial Intelligence (AI) and Mathematical (Linear and Integer Programming) Programming. The latter subject has grown up, to a large extent, independently of the former two. As a result the different emphases and assumed knowledge can make the recognition and comprehension of the connections difficult. Logic was developed originally largely for Philosophical and Foundational reasons. The development of Computational Logic is more recent arising from the ubiquitous nature of Computing and the ambitions of AI. Mathematical Programming (MP), by contrast, comes from Operational Research where Linear Programming (LP) models arise in many Resource Allocation contexts such as Production and Distribution. Integer Programming (IP) models are necessary to model the economic problems of discreteness and non-convexity. The majority of practical applications of IP restrict the integer variables to be 0 or 1. There are clear links between such 0-1 IP models and the Propositional Calculus (Boolean Algebra). Computational methods applicable to one are also applicable to the other and vice versa. The main aim of this paper is to survey and help unify the subjects as well as to explain the different concepts. Rather than attempt the impossible task of referring to all the published work in such diverse areas we give a comprehensive, but selected, set of references to provide leads into the different 0377-2217/95/$09.50 © 1995 Elsevier Science B.V. All rights reserved SSDI 0377-2217(93)E0359-6
606
H.P. Williams~European Journal of Operational Research 81 (1995) 605-616
approaches. An earlier version (Williams [35]) of this paper was presented at the Symposium on Applied Mathematical Modelling and Optimisation held at Brunel University in January 1991. In Section 2 we describe some computational problems arising in Logic which can be treated by IP models. These problems also provide insight into the nature and difficulty of IP. They include the problem of obtaining a minimal representation for a logic expression which is applied in Computer Circuit design. The methods used, based on a method of the Philosopher Quine [25] and [26], are forerunners of the method of Resolution due to Robinson [27] and [28] used in Logic languages such as P R O L O G . Other problems we consider concern computational procedures for decidable "fragments" of arithmetic. These fragments are, the Theory o f Dense Linear Order and Additive Arithmetic. It is recognised that LP can be modelled within the first and IP within the second. Computational procedures due to Langford [21] and Presburger [24] respectively were later recognised to be equivalent to Fourier-Motzkin elimination for LP (explained in Williams [29]) and an extension for IP (explained in Williams [30]). Interestingly Resolution mentioned above can be regarded as an extension of the application of Fourier±Motzkin elimination when logic statements are represented by linear inequalities. All these results are discussed. In Section 3 we apply the methods of Logic to IP. Logic can be applied to IP in two respects. Firstly it can be used to aid modelling and representation. Secondly it can be applied algorithmically. Granot and H a m m e r [12] applied Boolean Algebra to the modelling and simplification of Pure IP problems, They essentially applied the methods of Quine (Consensus and Subsumation) to a Logic representation (known as a Resolvent) of an IP in Disjunctive Normal Form. Following an improved representation due to Balas [1] Wilson, [36] and [37] generalised the logical representation to what is called '/3-Resolution'. These methods aid the modelling, representation and simplification of IP models but do not, on their own, provide a viable commercial algorithm (although they are useful for inferential problems). McKinnon and Williams [23] used the idea of a/3-Resolvent to create a modelling language for IP based on nested representations of the described 'greater-than-or-equal' predicate. Again all these results are described. In Section 4 we describe the reverse approach of applying the methods of LP and IP to solving problems in Computational Logic. This seems very fruitful. It was shown in [31] and [33] that Inference problems in the Propositional Calculus (Boolean Algebra) could be formulated as IP models and IP methods (e.g. Branch and Bound and Cutting Planes) applied. These methods have advantages over the traditional logic methods' (Resolution and its extensions) which are described here.
2. Computational problems of logic Minimal logical expressions in the Propositional Calculus The following is a sentence expressed in Disjunctive Form (DF). X l . X 4 V X l . X 4 V X 2 . X 5 V X l . X 3 . X 4 V X2.X3.~,Y4 V X I . X 2 . X 3 . X 5
(1)
'.', ' v ', ' - ' are the logical connectives 'and', 'or', 'not' respectively ('.' is regarded as more binding than 'V'). X1, X 2 etc. are atomic propositions which can be True (T) or False (F). They (e.g. X 1) or their negation (e.g. X 1) are known as literals. When connected by '.' (e.g. X2.X 5) we have a clause. Clauses are then connected by ' V' (a disjunction) to give a DF. In many applications of Circuit Design it is desirable to obtain a logically equivalent sentence to (1) with the minimum number of literals. Quine's procedure [25] and [26] involves first applying Subsumation and dropping the subsuming clauses (e.g. X1.X3.X4 subsumes X l . X 4 and can therefore be dropped and then Consensus and appending the resultant clauses (e.g. X3.Xa,X 5 is the consensus of X2.X 5 and X2.X3.,~4). These
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
607
procedures are alternately applied until there is no further change in a formula. Justifications and proofs can be found in Quine's papers. After successively applying these procedures we obtain a disjunction of Prime Implicants. It can be verified that the above example gives X 1. X 4 V X 1. X 4 V -X2 "X5 V X 2 . X 3 . S 4 V X 1. X 2 . X 3 V X 3 . S 4 .X5 V X l . X 3 . X 5
(2)
where the individual clauses are the Prime Implicants, i.e. clauses implying the original sentence whose literals do not properly contain a shorter implying clause. T h e r e still remains the problem of finding the Minimal Disjunction of Prime Implicants, i.e. a disjunction of as few as possible of these Prime Implicants which is equivalent to the original sentence. This turns out itself to be an IP problem known as the Covering Problem (see, e.g. [2] and [7]). For the example the solution to the resultant covering problem gives Xl.X 4 V Xl.X 4
V X~2.X5 V X 1.x 2 . x 3
(3)
as an equivalent sentence to (1) with the least number of clauses.
Resolution The method of Resolution due to Robinson [27] and [28] can be seen to be equivalent to Consensus. Resolution is normally applied to sentences in Conjunctive Form (CF) (i.e. a Conjunctive of Disjunctive Clauses). An example is ( X 2 V.~5).(.~ 2 v X 3 v X 4 ) .
(4)
The resolution of these two disjunctive clauses is
X3 VX4V~,~5 .
(5)
Such a statement is implied by the original clauses. In the method of Resolution (used by, for example, P R O L O G ) one tries to prove a conclusion ny negating it and ultimately hoping to obtain a contradiction (empty clause) by Resolution. Using De Morgan's Law the negation of the sentence (4) is Xz.X5 V X 2 .X3.1Ya
(6)
which by Consensus allows us to obtain the implicant X 3. X 4. X 5
(7)
i.e. the negation of (5), (7) therefore implies (6). These procedures (Resolution and Consensus are logical duals of each other.) It will be pointed out below that the procedure of Resolution can be viewed in an IP context as a specialisation of Fourier-Motzkin Elimination to particular types of IP constraint.
Decision procedures for fragments of mathematics If Arithmetic is formalised in the Predicate Calculus it is well known, by a result of G6del [11], that it is undecidable, i.e. there is no algorithm that will decide the truth or falsity of all sentences expressed in this Formal System. Some Fragments of Mathematics are, however, decidable. One such Fragment is the Theory of Dense Linear Order. This theory allows one to formalise inequality relations between real (or rational) numbers. A Decision Procedure for the Theory was produced by Langford [26].
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
608
Williams [29] pointed out that the LP problem could be formally stated within this Theory. Some time later it was realised that Langford's decision procedure, when applied to LP in this way, was the same as Fourier-Motzkin Elimination first proposed (for non-optimisation problems) by Fourier [9]. The method is fully explained in Williams [32]. It involves the successive elimination of (continuous) variables between inequalities. We consider all pairs of (like) inequalities in which the elimination variable (x) has opposite sign, e.g.
i ~ I, - a j x < g u, j ~ J ,
(8)
aix <--fi,
(9)
where ai, aj > O, 14= ~, J4: ~ and fi and gj are expressions involving the other variables and right-hand-sides. Formally (8) and (9) can be written as
3x[-gJaj
i~I,
j~J].
(10)
Since x lies in a continuum (e.g. rationals or reals) it can be shown that (10) is equivalent to i~I,
j~J,
(11)
O
j~J.
(12)
--gj/aj<_fi/ai,
written more simply as
Eq. (11) follows from (10) immediately. To show that (10) follows from (11) we can take x = Min {fi/ai} i~I
(13)
x=
(14)
or
The method does, o f course in general, prove unwieldy due to the increase of inequalities after each elimination. Some computational refinements are possible. Another Fragment of Arithmetic for which a decision procedure can be found is Additive Arithmetic obtained by dropping Multiplication. This does not present a restriction for IP since linear expressions involving multiplication by an integer can be expressed as successive addition, e.g. 2x is x + x. We do not need terms such as xy. This Fragment was shown to be decidable by a procedure due to Presburger [24]. Although concerned with equality systems it is straightforward to introduce inequality relations into the system. Then it is possible to formalise the IP problem. Presburger's procedure can be adapted to apply to these inequality systems and amounts to eliminating integer variables between all pairs of inequalities of the form (8) and (9). In this case system (10) is not equivalent to (11) (or (12)) since (10) implies the stronger condition that a n integer must lie between each of the expressions - g J a j and fi/ai. This condition can be stated in a finite manner as a generalisation of (12) by
V
air <_ajf i + ai&,&-r(mod aj),
r~{O, 1 , . . . , a j -
i ~ I,
j ~j.
(15)
i~I,
j~J.
(16)
1}
An alternative form of (15) is
V r~{O, 1 . . . . .
aj<_ajfi+aigj.fi-r(modai), ai-1}
Clearly (15) and (16) are alternative strengthenings of the condition (12) involving disjunctions of inequalities and congruences. Which is the more economical representation depends on the relative magnitudes of a i and aj.
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
609
In this way all the variables of an IP model can be eliminated to solve it. Again the method will, in general, be computationally prohibitively costly. If, however, one or other of the coefficients a i or aj is unity then (16) or (15) respectively reduce to (12) making for a simplification of Presburger's procedure. This simplification is described in Williams [34]. It is applicable, for example, to inference problems only involving H o r n Clauses and is discussed in Section 4. There is another interpretation of Fourier-Motzkin elimination which is worth remarking on and which is discussed in [32] and [34]. In the LP case one is projecting a convex polytope down into a lower dimension. The result is still a convex polytope expressed in terms of (in)equalities. For the IP extension one is projecting the lattice of integer points within a convex polytope down into a lower dimension. The result will generally be a proper subset of the lattice within the projected polytope. The subset still has a structure (that o f an integer monoid) but the more complex nature of this structure requires ,further stipulations. In special cases (e.g. Horn Clause systems), however, the complete lattice within the polytope is still applicable leading to a simplified representation.
3. Logic applied to integer programming Granot and H a m m e r [11] showed that the constraints of a Pure 0-1 IP model could be expressed in a logical form. For example, the following constraint: 5x 1 + 3x 2 + x 3 -}-X4 q-X 5 _~ 5,
X i E {0, 1},
(17)
could be stated in logical form as (~eY1V ~'Y2)-(~eY1V~eY3).(X 1V~eY4).(.eY1V ~'Y5).(~eY2 V~'Y3 V X 4 V.eY5)
(18)
where X i is the proposition 'X i = 1'. Each of the 5 disjunctive clauses in this conjunction capture the impossibility of certain variables simultaneously taking the value 1. It is more usual to consider the negation of (18) which is (by De Morgan's Laws) in DF. Xl.X 2 VXI.X 3 VXI.X 4 VXI.X 5 VX2.X3.X4.X
5.
(19)
This statement is known as the Resolvent of the constraint. The resolvents of all the constraints of a model can be joined b y disjunctions to give the Resolvent of the model, i.e., a statement whose falsity corresponds to a feasible solution. Hence the Resolvent is an alternative way of representing the constraints of a model. Whatsmore the simplification methods of Quine, mentioned earlier; can be applied to give a simplified representation. Such approaches to simplifying models are sometimes incorporated in Presolve procedures. They also provide a way of generating particular classes of cutting plane. Each of the clauses in (19) (or equivalently (18)) could alternatively be expressed as an IP constraint in its own right, e.g. the fifth clause gives X2 -]-X3 -]-X4 -]-X5 ~ 3,
(20)
stipulating the mutual incompatibility of X2, X3, X 4 and X 5. It was shown by Balas [1] that such a constraint could be strengthened by including those variables whose coefficients are greater-than-or-equal to the maximum coefficient of the included variables in the original constraint. Since the maximum such coefficient is 3, being that of x 2 in (17), we can also include x I (having a coefficient of 5) to give X 1 -]-X2 q-X 3 "}-X4 "]-X5 _~ 3.
(21)
Constraint (21) is 'stronger' than (20) in the sense that (21) implies (20) but not vice versa. It can be shown that so long as the constraint (21) is not dominated b y another c o n s t r a i n t it gives a facet of the
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
610
resultant Knapsack Polytope and hence is of advantage when LP relaxations form part of the IP algorithm. The disadvantage of these stronger constraints, however; when using a logical approach is that they do not have a natural logical representation. Whereas (20) is naturally represented by X 2 . X 3 . X 4 . X 5 = False
(22)
(21) would need the cumbersome disjunction X 1 . X z . X 3 . X 4 v X 1 . X 2 . X 3 . X 5 V X 1 . X 2 . X 4 . X 5 v X I . X 3 . X 4 . X 5 V X z . X 3 . X 4 . X 5 = False.
(23)
The space requirements of such logical representations would defeat their strength. One way of overcoming this defect is described by McKinnon and Williams [23] in developing a modelling language for IP. This centres round putting models into a Normal Form using (possibly nested) 'greater-than-orequal' predicates (ge). ge(n, {P1, P2 . . . . . Pr}) means 'n or more of propositions P1, P2,-.-, Pr are true'. (21) could then be written as . . . . .
This turns out to be a convenient modelling device for many problems. Even if it is not the most natural first statement of a condition it provides a transitional Normal Form that can be used to create an IP model. The process can then be automated computationally. As an example of its application it can be verified that the following proposition: a <2vb>3oe<8vf>7
(25)
can be expressed as g e ( 2 , {ge(1, {ge(2, {a > 2, b > 3}), e_< 8, f > 7}), g e ( 1 , {ge(2, {e > 8, f < 7}), a < 2, b < 3})}).
(26)
where a, b, e, f are some quantities (e.g. LP activities) and ' ~ ' means 'if and only if' and is less binding than ' v '. Similar ideas have also been used by Hadjiconstantinou, Lucas, Mitra and Moody [13] to convert logical statements into IP constraints. They use reverse Polish notation as an alternative to the 'greater-than-or-equal' predicate. Hiirlimann [18] also uses these ideas in the construction of a modelling language. The use of logic to provide algorithms for IP has been less successful although there is still (in the author's opinion) great potential. Granot and Hammer's procedure can be extended into an algorithm by placing a trial bound on the objective function. This gives rise to a constraint for which a Resolvent can be formed and incorporated i n t o the full Resolvent. I f simplification of the Resolvent by Quine's procedure (already shown to be equivalent to logical Resolution) shows it to be true, the objective value represented by the bound is not achievable, progressive weakening of the bound enables the optimal solution to be discovered. A procedure of Hooker [15] could be used to advantage here where additional conditions can be efficiently incorporated into the simplification. A generalisation of the traditional connectives of the Propositional Calculus (Boolean Algebra) similar to that of the 'greater-than-or-equal' predicate was considered and used by Wilson [36] and [37] in an algorithmic context. H e extended the conjunctive clauses of type (22) to give, e.g. for (21), a 'clause'.
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
611
'4 or more of X1, X2, X3, X 4, X 5 cannot be simultaneously true' The methods of Quine can then be generalised to simplify the resulting, so called, '/3-resolvents'. As would be expected the resultant rules become more detailed. Hooker [16] also discussed this idea and shows the greater power of such 'Generalised Resolution'.
4. Integer Programming applied to Logic There is a great interest in designing efficient computational procedures for solving problems of logical inference. Such problems arise in particular, in the field of Artificial Intelligence (AI) and the design of Expert Systems. As the size and complexity of problems tackled grows one faces potential 'combinatorial explosions' in solving such problems using established methods of Computational Logic (e.g. Resolution and its extensions). Two inferential problems often tackled are: (i) Is a certain conclusion a valid deduction from the premises? (ii) Is a set of statements consistent? Such problems are usually expressed within a derivative of the Predicate Calculus where statements of the form P(a, b, ... ) have a truth value which depends on the assignments (interpretations) given to the variables a, b, .... In order to solve these inferential problems one also has to lnstantiate these variables i.e., give them interpretations in terms of other objects. Once this instantiation has been done the resulting problem will have been reduced to one of the Propositional Calculus and the discussion of this paper will be relevant. The problem of Instantiation will generally be a substantial one. In theory one could try every possible interpretation of the variables but in practice this may be prohibitively costly. Jeroslow [19] has suggested how this process might be reduced and IP methods exploited, particularly if the variables can be given a spatial interpretation. Chandra and Hooker [4] explain this. Problem (ii) is the well known Satisfiability problem of Logic whose, to date, exponential complexity forms the basis for defining the complexity of other combinatorial problems. Resolution
We will begin by considering the following example and solving it by Resolution.
Example 1. Is the following deduction valid? Premises:
(i) (X 1 v X2)__~ (X 3 v X4); (ii) X 3 o X 1 , X 4 ; (iii) X 2 ~ (X 1 v X3); (iv) X4 V (X2.X32. Conclusion: (v) (X1 v X 4 ) . ( X : V-'Y4).(X3 VX4).(X1 v X a VX3). In order to use Resolution we must express all the premises and the negated conclusion as disjunctive clauses. This gives Premises: (i) X 1 V X 3 V X4; (ib) X 2 V X 3 V X4; (iia) X1VX3; (iib) X3 VX4; (iic) X1 VX3 vX4; (iiia) X 1 v X2 vX3; (iva) X2 V S4; (ivb) X 3 V X 4.
612
H.P. Williams~European Journal of Operational Research 81 (1995) 605-616
\_/ X4
\
/1VX4
x4\ /4 []
Fig. 1. Resolution Tree.
Negated Conclusion:
(va) X 1 V X 2 V X 3 ; (vb) X 1 V X4; (VC) X 2 V X 4 ;
(vd) X 3 V X 4. (Note that clause (iic) is redundant being a repeat of (ia).) T h e Resolution t r e e given in Fig. 1 shows how an empty clause (contradiction) can be obtained demonstrating that the deduction is valid. This procedure will in general be of exponential complexity. Note that it has at one stage been necessary to resolve a previously derived clause (X4) with the currently derived clause. This gives a clue to the potential combinatorially lengthy search which may be necessary. Branch-and-bound
As an alternative to Resolution we will formulate the example as an IP. This was suggested by Williams [31]. The original four premises correspond to the IP constraints (i)
(ii)
--X 1 - - X 2 q- 2X 3 + 2X 4 >_ 0, -x
1
-}-X3 "}-X4 ~ 0,
(27)
(28)
X1 -- 2X 3 --X 4 ~_~ --1,
(29)
--X 3 > --2,
(30)
(iii)
--X 1 --X 2
(iv)
X 2 + X 3 -- 2X 4 --> 0.
(31)
The conclusion corresponds to (V)
--X 1 - - X 2 - - X 3 -- 2 X 4 > - - 2 .
In order to test if the deduction is valid we may m i n i m i s e the expression in (32), i.e. - x 1 - subject to the constraints (27)-(31) and x i ~ {0, 1}.
(32) X 2 -- X 3 --
2X 4
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
613
If the Branch-and-Bound algorithm is used with LP Relaxations as a bounding mechanism we begin by solving the LP relaxation of this model. This gives X 1 ~--- 1,
X2 = 2,
X 3 = 53_, X 4 = ~ 4 '
objective = - - 3 ~3.
(33)
We cannot immediately decide if the conclusion, as represented by (32), follows since the LP relaxation objective is smaller than - 2 . It could of course happen (and frequently does in inference problems) that the LP optimal solution satisfies the objective constraint so proving the deduction. In this case, however, we must proceed to a genuine IP algorithm. The example can be solved to optimality after 6 nodes giving the optimal IP solution X 1 -- 1,
X2 = 0,
X 3 = 1,
X4 = 0,
Objective = - 2
(34)
so demonstrating the validity of the deduction. The procedure has a number of advantages over Resolution which may be enumerated. (i) It is not necessary to re-express the problem as a set of disjunctive clauses (clausal form). (ii) The deduction may possibly be proved by solving the computationally cheap LP relaxation. This is even more likely if a 'tighter' IP formulation is used. That above was purposely 'loose' in order to demonstrate the possible need for the Branch-and-Bound tree search phase. (iii) The use o f an objective function to represent the conclusion provides a 'goal' for the search removing the arbitrary nature of a Resolution search. Further discussion of the approach to solving inference problems is contained in [33]. Blair, Jeroslow and lmwe [3] give computational comparisons of Branch-and-Bound over Resolution and its extensions and suggest specialisations of Branch-and-Bound, based on the Davis-Putnam [8] algorithm, to incorporate some of the desirable features of logical methods. Chandra and Hooker [4] also discuss comparisons.
Horn Clauses Many logical inference systems (e.g. P R O L O G ) restrict the logical premises to a special form known as Horn Clauses. They must take one of the forms Xl.X2...
S n ----~Xn+l,
(35)
or
X 1 . X 2 . . . X , ~ [3,
(36)
where the Xi's are unnegated literals and ' D' represents an empty statement ( = False). As disjunctive clauses (35) and (36) are expressed as "~1VS2
V '''
VS n VXn+ 1
(37)
and
X1 VX2 V " " V .,Y,.,
(38)
respectively. Hence a Horn Clause is a disjunctive clause containing at most one unnegated literal. If all statements are H o r n Clauses then Resolution takes on a particularly simple form (see, e.g. Loveland [22]). It is only necessary to resolve each derived clause with one of the original (input) clauses. This is known as Linear Input Resolution and enables the resolution search to be severely reduced. We never have the situation demonstrated in Fig. 1 where we had to resolve two non-input clauses together. It is instructive to see how the Branch-and-Bound method, based on LP relaxations, specialises (in fact, can be avoided) for Horn Clause systems.
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
614
m
__
2vs
\_/
X 2 V ~'Y3
X3 Fig. 2. A linear input Resolution Tree.
Example 2. Is the following deduction valid? Premises: (i) X 1 V S 2 ; (ii) X 1 V -~2; (iii) X_2 V,,g3; Conclusion: (iv) X 3. All the premises and the negated conclusion are Horn. A rather trivial example has deliberately been chosen in order to illustrate a number of features of Horn systems very briefly. The Resolution tree is given in Fig. 2. The premises and conclusion give rise to the following inequalities among 0-1 variables (i)
x, - x 2 > 0,
(39)
(ii) (iii)
- X l - x2 > - 1,
(40)
X 2 - - X 3 >---0,
(41)
- x 3 > 0.
(42)
(iv)
If we minimise the left-hand-side of (42) subject to constraints (39) to (41) the LP relaxation gives the solution 1 (43) Objective = - 3. XI ~ X 2 ~ X 3 = ~ This solution, although fractional, immediately tells us that the optimal IP objective value must be at least 0 so satisfying constraint (42) and proving the deduction. It can be shown that this rounding property will always apply if the premises are Horn. Therefore it is unnecessary to proceed further with the Branch-and-Branch algorithm to find the IP optimum. In order to show why this property holds we will compare Fourier-Motzkin elimination with Resolution. We began, in Fig. 2 by resolving the two clauses X 1 V )~2
(44)
21V2~
(45)
and to obtain ~'Y2. The inequality equivalents of (44) and (45) together with other constraints involving x 1 are X 1 - - X 2 ~ 0,
--XI--X 2 ~ --1,
(46) (47) (48)
X 1 ~ 0,
(49)
--X 1 ~ --1.
(50)
H.P. Williams~European Journal of Operational Research 81 (1995) 605-616
615
Eliminating x 1 between all inequalities where it has opposite signs results in 3 redundant inequalities and -- 2X 2 _> -- 1,
(51)
Although x , is an integer variable this elimination is valid since at least one of the coefficients of x 1 is + 1 in each pair. T h e relations (15) or (16) reduce to (12). I f we divided through (51) by 2 and rounded we could obtain the inequality equivalent of (46) at this stage. W e choose, however, not to do this since we can use linear input resolution to eliminate x 2 between (51) and (41). (41) being an input inequality has a coefficient of ___1 for x 2 so again allowing an elimination of an integer variable without resultant congruences. T h e resultant inequality is - 2x 3 > - 1.
(52)
Some other redundant inequalities, which we ignore, arise from the bounds of 0 and 1 on x 2. Having reached the final stage we now divide through (52) by 2 and round to produce - x 3 > 0,
(53)
the desired conclusion. One rounding operation only will be necessary to obtain the optimal value of z and prove or refute deducibility. Since our procedure is a special case of trying to determine satisfiability it is worth pointing out how F o u r i e r - M o t z k i n elimination demonstrates another important result. If, after all eliminations we can deduce an LP (fractional) solution to a system then we can alternatively use expressions (13) or (14) and work backwards to deduce an integer solution since by linear input resolution at least one of a i or aj will be 1. We summarise these two strong results which apply to H o r n Clause systems. 1. Deduction can be proved in the inequality form with at most one rounding operation. 2. If a set of clauses in inequality form is satisfiable by fractional values between 0 and 1 it is satisfiable by integer (0-1) values. T h e first result is also proved by Jeroslow and Wang [20]. T h e r e is a very important result due to Chvfital [6] that any pure IP model can be solved by a combination of adding constraints, dividing by c o m m o n factors and rounding. These rounding operations may be nested, i.e. the same inequality added in at different stages and rounding applied. For H o r n Clause systems we only need o n e rounding operation applied at the end. The second result was proved by Blair, Jeroslow and Lowe [3]. Chandra and H o o k e r [5] generalise the class of H o r n clauses to larger systems having similar properties. Gallo and Urbani [10] show that there is advantage to be gained in relaxing t h e general satisfiability problem to the H o r n Clause satisfiability problem to provide a more efficient algorithm.
References [1] Balas, E., "Facets of the knapsack polytope", Mathematical Programming 8 (1975) 146-164. [2] Balas, E., and Padberg, M.W., "on the Set Covering Problem II", Operations Research 23 (1975) 74-80. [3] Blair, C.E., Jeroslow, R.G. and Lowe, J.IC "Some results and experiments in programming techniques for propositional logic", Computers & Operations Research 13 (1988) 633-645. [4] Chandra, V., and Hooker, J.N., "Logical Inference: A mathematical programming perspective", CC-88-24, School of Industrial Engineering, Purdue University, USA, 1988.
616
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
[5] Chandra, V., and Hooker, J.N., "Extended Horn sets in propositional logic", CC-89-2, School of Industrial Engineering, Purdue, University, USA, 1989. [6] Chv~tal, V., "Edmonds polytopes and hierarchy of combinatorial problems", Discrete Mathematics 4 (1973) 305-337. [7] Cobham, A., Fridshal, R., and North, J.H., "An application of linear programming to the minimisation of Boolean functions", IBM Research Report RC-472, Yorktown Heights, 1961. [8] Davis, M., and Putnam, H., "A computing procedure for quantification theory", Journal of the ACM 7 (1960) 201-215. [9] Fourier, J.B.J., "Solution d'une question particuli~re du calcul des An6galit6s", in: Oevres H, Paris, 1826, 314-328. [10] Gallo, G., and Urbani, G., "Algorithms for testing the satisfiability of propositional formulae", Journal of Logic Programming 7 (1989) 45-61. [11] G6del, K., On Undecidable Propositions of Formal Mathematical Systems, Princeton University Press, Princeton, NY, 1934. [12] Granot, F., and Hammer, P.L., "On the use of the Boolean functions in 0-1 Programming", Methods of Operations Research 12 (1971) 154-184. [13] Hadjiconstantinou, E., Lucas, C., Mitra, G., and Moody, S., "Tools for reformulating logical forms into zero-one mixed integer programs", European Journal of Operational Research 72 (1994) 263-277. [14] Hooker, J.N., "A quantitative approach to logical interference", Decision Support Systems 4 (1988) 45-69. [15] Hooker, J.N., "Solving the incremental satisfiability problem", Working Paper 1991-9, Graduate School of Industrial Administration, Carnegie-Mellon University, 1991. [16] Hooker, J.N, "Generalised resolution and cutting planes", Annals of Operations Research 12 (1988) 214-239. [17] Hooker, J.N., "Input Proofs and rank one cutting planes", ORSA Journal on Computing, to appear. [18] Hiirlimann, T., "IP, MIP and Logical Modelling using LPL", Paper presented at Symposium on Applied Mathematical Modelling and Optimisation, Budapest, January 1993. [19] Jeroslow, R.G., "Logic-based decision support", Annals of Discrete Mathematics 40 (1989). [20] Jeroslow, R.G., and Wang, J., "Dynamic Programming, integral polyhedra and Horn Clause knowledge bases", Working Paper, Georgia Institute of Technology, Atlanta, GA, 1987. [21] Langford, C.H., "Some theorems on deducibility", Annals of Mathematics 1 28 (1927) 16-40. [22] Loveland, D.W., Automated Theorem Proving: A Logical Basis, North-Holland, Amsterdam, 1978. [23] McKinnon, K.I.M., and Williams, H.P., "Constructing integer programming models by the predicate calculus", Annals of Operations Research' 21 (1989) 227-246. [24] Presburger, M., "Uber die Vollst5ndigkeit eines Gewissen Systems der Arithmetik ganzer Zahlen in Welchem die Addition als einzige Operation hervortritit", in: Comptes-Residus du I Congres de Mathematicians de pays Slaves, Warsaw, 1930, 92-101. [25] Quine, W.V., "The problem of simplifying truth functions", American Mathematical Monthly 59 (1952) 521-531. [26] Quine, W.V., "A way to simplify truth functions", Americal Mathematical Monthly 62 (1955) 627-631. [27] Robinson, J.A., "A machine-oriented logic based on the resolution principle", Journal of the ACM 12 (1965) 23-41. [28] Robinson, J.A., "A generalised resolution principle", Machine Intelligence 3 (1968) 77-93. [29] Wiliams, H.P., "An algorithm for the solution of linear programming problems", IBM Working Paper, 1970. [30] Williams, H.P., "Fourier-Motzkin elimination extension to integer programming problems", Journal of Combinatorial Theory 21 (1976) 118-123. [31] Williams, H.P., "Logical problems and integer programming", Bulletin of the IMA 13 (1977) 18-20. [32] Williams, H.P., "Fourier's method of linear programming and its dual", American Mathematical Monthly 93 (1986) 681-695. [33] Williams, H.P., "Linear and integer programming applied to the propositional calculus", International Journal of Systems Research and Information Science 2 (1987) 81-100. [34] Williams, H.P., "The elimination of integer variables", Journal of the Operations Research Society 43 (1992) 387-393. [35] Williams, H.P., "Computational logic and integer programming: Connections between the methods of logic, AI and OR", Paper presented at Symposium on Applied Mathematical Modelling and Optimisation, Brunel University, January 1991. [36] Wilson, J.M., "Methods of (0-1) integer programming using a Boolean Approach", D. Phil. Thesis, University of Sussex, 1975. [37] Wilson, J.M., "A method of (0-1) integer programming using a Boolean approach", in: T.B. Boffey (ed.) Proceedings of CP77 University of Liverpool, 1977.
European Journal of Operational Research 81 (1995) 605-616
Theory and Methodology
Logic applied to integer programming and integer programming applied to logic H.P. Williams Faculty of Mathematical Studies, University of Southampton, Southampton, S09 5NH, UK Received July 1993; revised November 1993
Abstract
The many connections between the methods of Computational Logic and Integer Programming (IP) are surveyed. It is shown how computational problems arising in formal logic can be solved by IP. Also it is shown how the methods of logic are applicable both to modelling and solving IP models. It is shown how Fourier-Motzkin elimination for Linear Programming, when specialised to 0-1 IP models gives rise to the logic method of Resolution. Finally conventional IP methods are applied to solving logical inference problems.
Keywords: Fourier-Motzkin elimination; Logical deduction; Logical resolution; Propositional calculus; Satisfiability; 0-1 integer programming
1. Introduction
There are many cross connections between Logic, Artificial Intelligence (AI) and Mathematical (Linear and Integer Programming) Programming. The latter subject has grown up, to a large extent, independently of the former two. As a result the different emphases and assumed knowledge can make the recognition and comprehension of the connections difficult. Logic was developed originally largely for Philosophical and Foundational reasons. The development of Computational Logic is more recent arising from the ubiquitous nature of Computing and the ambitions of AI. Mathematical Programming (MP), by contrast, comes from Operational Research where Linear Programming (LP) models arise in many Resource Allocation contexts such as Production and Distribution. Integer Programming (IP) models are necessary to model the economic problems of discreteness and non-convexity. The majority of practical applications of IP restrict the integer variables to be 0 or 1. There are clear links between such 0-1 IP models and the Propositional Calculus (Boolean Algebra). Computational methods applicable to one are also applicable to the other and vice versa. The main aim of this paper is to survey and help unify the subjects as well as to explain the different concepts. Rather than attempt the impossible task of referring to all the published work in such diverse areas we give a comprehensive, but selected, set of references to provide leads into the different 0377-2217/95/$09.50 © 1995 Elsevier Science B.V. All rights reserved SSDI 0377-2217(93)E0359-6
606
H.P. Williams~European Journal of Operational Research 81 (1995) 605-616
approaches. An earlier version (Williams [35]) of this paper was presented at the Symposium on Applied Mathematical Modelling and Optimisation held at Brunel University in January 1991. In Section 2 we describe some computational problems arising in Logic which can be treated by IP models. These problems also provide insight into the nature and difficulty of IP. They include the problem of obtaining a minimal representation for a logic expression which is applied in Computer Circuit design. The methods used, based on a method of the Philosopher Quine [25] and [26], are forerunners of the method of Resolution due to Robinson [27] and [28] used in Logic languages such as P R O L O G . Other problems we consider concern computational procedures for decidable "fragments" of arithmetic. These fragments are, the Theory o f Dense Linear Order and Additive Arithmetic. It is recognised that LP can be modelled within the first and IP within the second. Computational procedures due to Langford [21] and Presburger [24] respectively were later recognised to be equivalent to Fourier-Motzkin elimination for LP (explained in Williams [29]) and an extension for IP (explained in Williams [30]). Interestingly Resolution mentioned above can be regarded as an extension of the application of Fourier±Motzkin elimination when logic statements are represented by linear inequalities. All these results are discussed. In Section 3 we apply the methods of Logic to IP. Logic can be applied to IP in two respects. Firstly it can be used to aid modelling and representation. Secondly it can be applied algorithmically. Granot and H a m m e r [12] applied Boolean Algebra to the modelling and simplification of Pure IP problems, They essentially applied the methods of Quine (Consensus and Subsumation) to a Logic representation (known as a Resolvent) of an IP in Disjunctive Normal Form. Following an improved representation due to Balas [1] Wilson, [36] and [37] generalised the logical representation to what is called '/3-Resolution'. These methods aid the modelling, representation and simplification of IP models but do not, on their own, provide a viable commercial algorithm (although they are useful for inferential problems). McKinnon and Williams [23] used the idea of a/3-Resolvent to create a modelling language for IP based on nested representations of the described 'greater-than-or-equal' predicate. Again all these results are described. In Section 4 we describe the reverse approach of applying the methods of LP and IP to solving problems in Computational Logic. This seems very fruitful. It was shown in [31] and [33] that Inference problems in the Propositional Calculus (Boolean Algebra) could be formulated as IP models and IP methods (e.g. Branch and Bound and Cutting Planes) applied. These methods have advantages over the traditional logic methods' (Resolution and its extensions) which are described here.
2. Computational problems of logic Minimal logical expressions in the Propositional Calculus The following is a sentence expressed in Disjunctive Form (DF). X l . X 4 V X l . X 4 V X 2 . X 5 V X l . X 3 . X 4 V X2.X3.~,Y4 V X I . X 2 . X 3 . X 5
(1)
'.', ' v ', ' - ' are the logical connectives 'and', 'or', 'not' respectively ('.' is regarded as more binding than 'V'). X1, X 2 etc. are atomic propositions which can be True (T) or False (F). They (e.g. X 1) or their negation (e.g. X 1) are known as literals. When connected by '.' (e.g. X2.X 5) we have a clause. Clauses are then connected by ' V' (a disjunction) to give a DF. In many applications of Circuit Design it is desirable to obtain a logically equivalent sentence to (1) with the minimum number of literals. Quine's procedure [25] and [26] involves first applying Subsumation and dropping the subsuming clauses (e.g. X1.X3.X4 subsumes X l . X 4 and can therefore be dropped and then Consensus and appending the resultant clauses (e.g. X3.Xa,X 5 is the consensus of X2.X 5 and X2.X3.,~4). These
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
607
procedures are alternately applied until there is no further change in a formula. Justifications and proofs can be found in Quine's papers. After successively applying these procedures we obtain a disjunction of Prime Implicants. It can be verified that the above example gives X 1. X 4 V X 1. X 4 V -X2 "X5 V X 2 . X 3 . S 4 V X 1. X 2 . X 3 V X 3 . S 4 .X5 V X l . X 3 . X 5
(2)
where the individual clauses are the Prime Implicants, i.e. clauses implying the original sentence whose literals do not properly contain a shorter implying clause. T h e r e still remains the problem of finding the Minimal Disjunction of Prime Implicants, i.e. a disjunction of as few as possible of these Prime Implicants which is equivalent to the original sentence. This turns out itself to be an IP problem known as the Covering Problem (see, e.g. [2] and [7]). For the example the solution to the resultant covering problem gives Xl.X 4 V Xl.X 4
V X~2.X5 V X 1.x 2 . x 3
(3)
as an equivalent sentence to (1) with the least number of clauses.
Resolution The method of Resolution due to Robinson [27] and [28] can be seen to be equivalent to Consensus. Resolution is normally applied to sentences in Conjunctive Form (CF) (i.e. a Conjunctive of Disjunctive Clauses). An example is ( X 2 V.~5).(.~ 2 v X 3 v X 4 ) .
(4)
The resolution of these two disjunctive clauses is
X3 VX4V~,~5 .
(5)
Such a statement is implied by the original clauses. In the method of Resolution (used by, for example, P R O L O G ) one tries to prove a conclusion ny negating it and ultimately hoping to obtain a contradiction (empty clause) by Resolution. Using De Morgan's Law the negation of the sentence (4) is Xz.X5 V X 2 .X3.1Ya
(6)
which by Consensus allows us to obtain the implicant X 3. X 4. X 5
(7)
i.e. the negation of (5), (7) therefore implies (6). These procedures (Resolution and Consensus are logical duals of each other.) It will be pointed out below that the procedure of Resolution can be viewed in an IP context as a specialisation of Fourier-Motzkin Elimination to particular types of IP constraint.
Decision procedures for fragments of mathematics If Arithmetic is formalised in the Predicate Calculus it is well known, by a result of G6del [11], that it is undecidable, i.e. there is no algorithm that will decide the truth or falsity of all sentences expressed in this Formal System. Some Fragments of Mathematics are, however, decidable. One such Fragment is the Theory of Dense Linear Order. This theory allows one to formalise inequality relations between real (or rational) numbers. A Decision Procedure for the Theory was produced by Langford [26].
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
608
Williams [29] pointed out that the LP problem could be formally stated within this Theory. Some time later it was realised that Langford's decision procedure, when applied to LP in this way, was the same as Fourier-Motzkin Elimination first proposed (for non-optimisation problems) by Fourier [9]. The method is fully explained in Williams [32]. It involves the successive elimination of (continuous) variables between inequalities. We consider all pairs of (like) inequalities in which the elimination variable (x) has opposite sign, e.g.
i ~ I, - a j x < g u, j ~ J ,
(8)
aix <--fi,
(9)
where ai, aj > O, 14= ~, J4: ~ and fi and gj are expressions involving the other variables and right-hand-sides. Formally (8) and (9) can be written as
3x[-gJaj
i~I,
j~J].
(10)
Since x lies in a continuum (e.g. rationals or reals) it can be shown that (10) is equivalent to i~I,
j~J,
(11)
O
j~J.
(12)
--gj/aj<_fi/ai,
written more simply as
Eq. (11) follows from (10) immediately. To show that (10) follows from (11) we can take x = Min {fi/ai} i~I
(13)
x=
(14)
or
The method does, o f course in general, prove unwieldy due to the increase of inequalities after each elimination. Some computational refinements are possible. Another Fragment of Arithmetic for which a decision procedure can be found is Additive Arithmetic obtained by dropping Multiplication. This does not present a restriction for IP since linear expressions involving multiplication by an integer can be expressed as successive addition, e.g. 2x is x + x. We do not need terms such as xy. This Fragment was shown to be decidable by a procedure due to Presburger [24]. Although concerned with equality systems it is straightforward to introduce inequality relations into the system. Then it is possible to formalise the IP problem. Presburger's procedure can be adapted to apply to these inequality systems and amounts to eliminating integer variables between all pairs of inequalities of the form (8) and (9). In this case system (10) is not equivalent to (11) (or (12)) since (10) implies the stronger condition that a n integer must lie between each of the expressions - g J a j and fi/ai. This condition can be stated in a finite manner as a generalisation of (12) by
V
air <_ajf i + ai&,&-r(mod aj),
r~{O, 1 , . . . , a j -
i ~ I,
j ~j.
(15)
i~I,
j~J.
(16)
1}
An alternative form of (15) is
V r~{O, 1 . . . . .
aj<_ajfi+aigj.fi-r(modai), ai-1}
Clearly (15) and (16) are alternative strengthenings of the condition (12) involving disjunctions of inequalities and congruences. Which is the more economical representation depends on the relative magnitudes of a i and aj.
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
609
In this way all the variables of an IP model can be eliminated to solve it. Again the method will, in general, be computationally prohibitively costly. If, however, one or other of the coefficients a i or aj is unity then (16) or (15) respectively reduce to (12) making for a simplification of Presburger's procedure. This simplification is described in Williams [34]. It is applicable, for example, to inference problems only involving H o r n Clauses and is discussed in Section 4. There is another interpretation of Fourier-Motzkin elimination which is worth remarking on and which is discussed in [32] and [34]. In the LP case one is projecting a convex polytope down into a lower dimension. The result is still a convex polytope expressed in terms of (in)equalities. For the IP extension one is projecting the lattice of integer points within a convex polytope down into a lower dimension. The result will generally be a proper subset of the lattice within the projected polytope. The subset still has a structure (that o f an integer monoid) but the more complex nature of this structure requires ,further stipulations. In special cases (e.g. Horn Clause systems), however, the complete lattice within the polytope is still applicable leading to a simplified representation.
3. Logic applied to integer programming Granot and H a m m e r [11] showed that the constraints of a Pure 0-1 IP model could be expressed in a logical form. For example, the following constraint: 5x 1 + 3x 2 + x 3 -}-X4 q-X 5 _~ 5,
X i E {0, 1},
(17)
could be stated in logical form as (~eY1V ~'Y2)-(~eY1V~eY3).(X 1V~eY4).(.eY1V ~'Y5).(~eY2 V~'Y3 V X 4 V.eY5)
(18)
where X i is the proposition 'X i = 1'. Each of the 5 disjunctive clauses in this conjunction capture the impossibility of certain variables simultaneously taking the value 1. It is more usual to consider the negation of (18) which is (by De Morgan's Laws) in DF. Xl.X 2 VXI.X 3 VXI.X 4 VXI.X 5 VX2.X3.X4.X
5.
(19)
This statement is known as the Resolvent of the constraint. The resolvents of all the constraints of a model can be joined b y disjunctions to give the Resolvent of the model, i.e., a statement whose falsity corresponds to a feasible solution. Hence the Resolvent is an alternative way of representing the constraints of a model. Whatsmore the simplification methods of Quine, mentioned earlier; can be applied to give a simplified representation. Such approaches to simplifying models are sometimes incorporated in Presolve procedures. They also provide a way of generating particular classes of cutting plane. Each of the clauses in (19) (or equivalently (18)) could alternatively be expressed as an IP constraint in its own right, e.g. the fifth clause gives X2 -]-X3 -]-X4 -]-X5 ~ 3,
(20)
stipulating the mutual incompatibility of X2, X3, X 4 and X 5. It was shown by Balas [1] that such a constraint could be strengthened by including those variables whose coefficients are greater-than-or-equal to the maximum coefficient of the included variables in the original constraint. Since the maximum such coefficient is 3, being that of x 2 in (17), we can also include x I (having a coefficient of 5) to give X 1 -]-X2 q-X 3 "}-X4 "]-X5 _~ 3.
(21)
Constraint (21) is 'stronger' than (20) in the sense that (21) implies (20) but not vice versa. It can be shown that so long as the constraint (21) is not dominated b y another c o n s t r a i n t it gives a facet of the
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
610
resultant Knapsack Polytope and hence is of advantage when LP relaxations form part of the IP algorithm. The disadvantage of these stronger constraints, however; when using a logical approach is that they do not have a natural logical representation. Whereas (20) is naturally represented by X 2 . X 3 . X 4 . X 5 = False
(22)
(21) would need the cumbersome disjunction X 1 . X z . X 3 . X 4 v X 1 . X 2 . X 3 . X 5 V X 1 . X 2 . X 4 . X 5 v X I . X 3 . X 4 . X 5 V X z . X 3 . X 4 . X 5 = False.
(23)
The space requirements of such logical representations would defeat their strength. One way of overcoming this defect is described by McKinnon and Williams [23] in developing a modelling language for IP. This centres round putting models into a Normal Form using (possibly nested) 'greater-than-orequal' predicates (ge). ge(n, {P1, P2 . . . . . Pr}) means 'n or more of propositions P1, P2,-.-, Pr are true'. (21) could then be written as . . . . .
This turns out to be a convenient modelling device for many problems. Even if it is not the most natural first statement of a condition it provides a transitional Normal Form that can be used to create an IP model. The process can then be automated computationally. As an example of its application it can be verified that the following proposition: a <2vb>3oe<8vf>7
(25)
can be expressed as g e ( 2 , {ge(1, {ge(2, {a > 2, b > 3}), e_< 8, f > 7}), g e ( 1 , {ge(2, {e > 8, f < 7}), a < 2, b < 3})}).
(26)
where a, b, e, f are some quantities (e.g. LP activities) and ' ~ ' means 'if and only if' and is less binding than ' v '. Similar ideas have also been used by Hadjiconstantinou, Lucas, Mitra and Moody [13] to convert logical statements into IP constraints. They use reverse Polish notation as an alternative to the 'greater-than-or-equal' predicate. Hiirlimann [18] also uses these ideas in the construction of a modelling language. The use of logic to provide algorithms for IP has been less successful although there is still (in the author's opinion) great potential. Granot and Hammer's procedure can be extended into an algorithm by placing a trial bound on the objective function. This gives rise to a constraint for which a Resolvent can be formed and incorporated i n t o the full Resolvent. I f simplification of the Resolvent by Quine's procedure (already shown to be equivalent to logical Resolution) shows it to be true, the objective value represented by the bound is not achievable, progressive weakening of the bound enables the optimal solution to be discovered. A procedure of Hooker [15] could be used to advantage here where additional conditions can be efficiently incorporated into the simplification. A generalisation of the traditional connectives of the Propositional Calculus (Boolean Algebra) similar to that of the 'greater-than-or-equal' predicate was considered and used by Wilson [36] and [37] in an algorithmic context. H e extended the conjunctive clauses of type (22) to give, e.g. for (21), a 'clause'.
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
611
'4 or more of X1, X2, X3, X 4, X 5 cannot be simultaneously true' The methods of Quine can then be generalised to simplify the resulting, so called, '/3-resolvents'. As would be expected the resultant rules become more detailed. Hooker [16] also discussed this idea and shows the greater power of such 'Generalised Resolution'.
4. Integer Programming applied to Logic There is a great interest in designing efficient computational procedures for solving problems of logical inference. Such problems arise in particular, in the field of Artificial Intelligence (AI) and the design of Expert Systems. As the size and complexity of problems tackled grows one faces potential 'combinatorial explosions' in solving such problems using established methods of Computational Logic (e.g. Resolution and its extensions). Two inferential problems often tackled are: (i) Is a certain conclusion a valid deduction from the premises? (ii) Is a set of statements consistent? Such problems are usually expressed within a derivative of the Predicate Calculus where statements of the form P(a, b, ... ) have a truth value which depends on the assignments (interpretations) given to the variables a, b, .... In order to solve these inferential problems one also has to lnstantiate these variables i.e., give them interpretations in terms of other objects. Once this instantiation has been done the resulting problem will have been reduced to one of the Propositional Calculus and the discussion of this paper will be relevant. The problem of Instantiation will generally be a substantial one. In theory one could try every possible interpretation of the variables but in practice this may be prohibitively costly. Jeroslow [19] has suggested how this process might be reduced and IP methods exploited, particularly if the variables can be given a spatial interpretation. Chandra and Hooker [4] explain this. Problem (ii) is the well known Satisfiability problem of Logic whose, to date, exponential complexity forms the basis for defining the complexity of other combinatorial problems. Resolution
We will begin by considering the following example and solving it by Resolution.
Example 1. Is the following deduction valid? Premises:
(i) (X 1 v X2)__~ (X 3 v X4); (ii) X 3 o X 1 , X 4 ; (iii) X 2 ~ (X 1 v X3); (iv) X4 V (X2.X32. Conclusion: (v) (X1 v X 4 ) . ( X : V-'Y4).(X3 VX4).(X1 v X a VX3). In order to use Resolution we must express all the premises and the negated conclusion as disjunctive clauses. This gives Premises: (i) X 1 V X 3 V X4; (ib) X 2 V X 3 V X4; (iia) X1VX3; (iib) X3 VX4; (iic) X1 VX3 vX4; (iiia) X 1 v X2 vX3; (iva) X2 V S4; (ivb) X 3 V X 4.
612
H.P. Williams~European Journal of Operational Research 81 (1995) 605-616
\_/ X4
\
/1VX4
x4\ /4 []
Fig. 1. Resolution Tree.
Negated Conclusion:
(va) X 1 V X 2 V X 3 ; (vb) X 1 V X4; (VC) X 2 V X 4 ;
(vd) X 3 V X 4. (Note that clause (iic) is redundant being a repeat of (ia).) T h e Resolution t r e e given in Fig. 1 shows how an empty clause (contradiction) can be obtained demonstrating that the deduction is valid. This procedure will in general be of exponential complexity. Note that it has at one stage been necessary to resolve a previously derived clause (X4) with the currently derived clause. This gives a clue to the potential combinatorially lengthy search which may be necessary. Branch-and-bound
As an alternative to Resolution we will formulate the example as an IP. This was suggested by Williams [31]. The original four premises correspond to the IP constraints (i)
(ii)
--X 1 - - X 2 q- 2X 3 + 2X 4 >_ 0, -x
1
-}-X3 "}-X4 ~ 0,
(27)
(28)
X1 -- 2X 3 --X 4 ~_~ --1,
(29)
--X 3 > --2,
(30)
(iii)
--X 1 --X 2
(iv)
X 2 + X 3 -- 2X 4 --> 0.
(31)
The conclusion corresponds to (V)
--X 1 - - X 2 - - X 3 -- 2 X 4 > - - 2 .
In order to test if the deduction is valid we may m i n i m i s e the expression in (32), i.e. - x 1 - subject to the constraints (27)-(31) and x i ~ {0, 1}.
(32) X 2 -- X 3 --
2X 4
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
613
If the Branch-and-Bound algorithm is used with LP Relaxations as a bounding mechanism we begin by solving the LP relaxation of this model. This gives X 1 ~--- 1,
X2 = 2,
X 3 = 53_, X 4 = ~ 4 '
objective = - - 3 ~3.
(33)
We cannot immediately decide if the conclusion, as represented by (32), follows since the LP relaxation objective is smaller than - 2 . It could of course happen (and frequently does in inference problems) that the LP optimal solution satisfies the objective constraint so proving the deduction. In this case, however, we must proceed to a genuine IP algorithm. The example can be solved to optimality after 6 nodes giving the optimal IP solution X 1 -- 1,
X2 = 0,
X 3 = 1,
X4 = 0,
Objective = - 2
(34)
so demonstrating the validity of the deduction. The procedure has a number of advantages over Resolution which may be enumerated. (i) It is not necessary to re-express the problem as a set of disjunctive clauses (clausal form). (ii) The deduction may possibly be proved by solving the computationally cheap LP relaxation. This is even more likely if a 'tighter' IP formulation is used. That above was purposely 'loose' in order to demonstrate the possible need for the Branch-and-Bound tree search phase. (iii) The use o f an objective function to represent the conclusion provides a 'goal' for the search removing the arbitrary nature of a Resolution search. Further discussion of the approach to solving inference problems is contained in [33]. Blair, Jeroslow and lmwe [3] give computational comparisons of Branch-and-Bound over Resolution and its extensions and suggest specialisations of Branch-and-Bound, based on the Davis-Putnam [8] algorithm, to incorporate some of the desirable features of logical methods. Chandra and Hooker [4] also discuss comparisons.
Horn Clauses Many logical inference systems (e.g. P R O L O G ) restrict the logical premises to a special form known as Horn Clauses. They must take one of the forms Xl.X2...
S n ----~Xn+l,
(35)
or
X 1 . X 2 . . . X , ~ [3,
(36)
where the Xi's are unnegated literals and ' D' represents an empty statement ( = False). As disjunctive clauses (35) and (36) are expressed as "~1VS2
V '''
VS n VXn+ 1
(37)
and
X1 VX2 V " " V .,Y,.,
(38)
respectively. Hence a Horn Clause is a disjunctive clause containing at most one unnegated literal. If all statements are H o r n Clauses then Resolution takes on a particularly simple form (see, e.g. Loveland [22]). It is only necessary to resolve each derived clause with one of the original (input) clauses. This is known as Linear Input Resolution and enables the resolution search to be severely reduced. We never have the situation demonstrated in Fig. 1 where we had to resolve two non-input clauses together. It is instructive to see how the Branch-and-Bound method, based on LP relaxations, specialises (in fact, can be avoided) for Horn Clause systems.
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
614
m
__
2vs
\_/
X 2 V ~'Y3
X3 Fig. 2. A linear input Resolution Tree.
Example 2. Is the following deduction valid? Premises: (i) X 1 V S 2 ; (ii) X 1 V -~2; (iii) X_2 V,,g3; Conclusion: (iv) X 3. All the premises and the negated conclusion are Horn. A rather trivial example has deliberately been chosen in order to illustrate a number of features of Horn systems very briefly. The Resolution tree is given in Fig. 2. The premises and conclusion give rise to the following inequalities among 0-1 variables (i)
x, - x 2 > 0,
(39)
(ii) (iii)
- X l - x2 > - 1,
(40)
X 2 - - X 3 >---0,
(41)
- x 3 > 0.
(42)
(iv)
If we minimise the left-hand-side of (42) subject to constraints (39) to (41) the LP relaxation gives the solution 1 (43) Objective = - 3. XI ~ X 2 ~ X 3 = ~ This solution, although fractional, immediately tells us that the optimal IP objective value must be at least 0 so satisfying constraint (42) and proving the deduction. It can be shown that this rounding property will always apply if the premises are Horn. Therefore it is unnecessary to proceed further with the Branch-and-Branch algorithm to find the IP optimum. In order to show why this property holds we will compare Fourier-Motzkin elimination with Resolution. We began, in Fig. 2 by resolving the two clauses X 1 V )~2
(44)
21V2~
(45)
and to obtain ~'Y2. The inequality equivalents of (44) and (45) together with other constraints involving x 1 are X 1 - - X 2 ~ 0,
--XI--X 2 ~ --1,
(46) (47) (48)
X 1 ~ 0,
(49)
--X 1 ~ --1.
(50)
H.P. Williams~European Journal of Operational Research 81 (1995) 605-616
615
Eliminating x 1 between all inequalities where it has opposite signs results in 3 redundant inequalities and -- 2X 2 _> -- 1,
(51)
Although x , is an integer variable this elimination is valid since at least one of the coefficients of x 1 is + 1 in each pair. T h e relations (15) or (16) reduce to (12). I f we divided through (51) by 2 and rounded we could obtain the inequality equivalent of (46) at this stage. W e choose, however, not to do this since we can use linear input resolution to eliminate x 2 between (51) and (41). (41) being an input inequality has a coefficient of ___1 for x 2 so again allowing an elimination of an integer variable without resultant congruences. T h e resultant inequality is - 2x 3 > - 1.
(52)
Some other redundant inequalities, which we ignore, arise from the bounds of 0 and 1 on x 2. Having reached the final stage we now divide through (52) by 2 and round to produce - x 3 > 0,
(53)
the desired conclusion. One rounding operation only will be necessary to obtain the optimal value of z and prove or refute deducibility. Since our procedure is a special case of trying to determine satisfiability it is worth pointing out how F o u r i e r - M o t z k i n elimination demonstrates another important result. If, after all eliminations we can deduce an LP (fractional) solution to a system then we can alternatively use expressions (13) or (14) and work backwards to deduce an integer solution since by linear input resolution at least one of a i or aj will be 1. We summarise these two strong results which apply to H o r n Clause systems. 1. Deduction can be proved in the inequality form with at most one rounding operation. 2. If a set of clauses in inequality form is satisfiable by fractional values between 0 and 1 it is satisfiable by integer (0-1) values. T h e first result is also proved by Jeroslow and Wang [20]. T h e r e is a very important result due to Chvfital [6] that any pure IP model can be solved by a combination of adding constraints, dividing by c o m m o n factors and rounding. These rounding operations may be nested, i.e. the same inequality added in at different stages and rounding applied. For H o r n Clause systems we only need o n e rounding operation applied at the end. The second result was proved by Blair, Jeroslow and Lowe [3]. Chandra and H o o k e r [5] generalise the class of H o r n clauses to larger systems having similar properties. Gallo and Urbani [10] show that there is advantage to be gained in relaxing t h e general satisfiability problem to the H o r n Clause satisfiability problem to provide a more efficient algorithm.
References [1] Balas, E., "Facets of the knapsack polytope", Mathematical Programming 8 (1975) 146-164. [2] Balas, E., and Padberg, M.W., "on the Set Covering Problem II", Operations Research 23 (1975) 74-80. [3] Blair, C.E., Jeroslow, R.G. and Lowe, J.IC "Some results and experiments in programming techniques for propositional logic", Computers & Operations Research 13 (1988) 633-645. [4] Chandra, V., and Hooker, J.N., "Logical Inference: A mathematical programming perspective", CC-88-24, School of Industrial Engineering, Purdue University, USA, 1988.
616
H.P. Williams/European Journal of Operational Research 81 (1995) 605-616
[5] Chandra, V., and Hooker, J.N., "Extended Horn sets in propositional logic", CC-89-2, School of Industrial Engineering, Purdue, University, USA, 1989. [6] Chv~tal, V., "Edmonds polytopes and hierarchy of combinatorial problems", Discrete Mathematics 4 (1973) 305-337. [7] Cobham, A., Fridshal, R., and North, J.H., "An application of linear programming to the minimisation of Boolean functions", IBM Research Report RC-472, Yorktown Heights, 1961. [8] Davis, M., and Putnam, H., "A computing procedure for quantification theory", Journal of the ACM 7 (1960) 201-215. [9] Fourier, J.B.J., "Solution d'une question particuli~re du calcul des An6galit6s", in: Oevres H, Paris, 1826, 314-328. [10] Gallo, G., and Urbani, G., "Algorithms for testing the satisfiability of propositional formulae", Journal of Logic Programming 7 (1989) 45-61. [11] G6del, K., On Undecidable Propositions of Formal Mathematical Systems, Princeton University Press, Princeton, NY, 1934. [12] Granot, F., and Hammer, P.L., "On the use of the Boolean functions in 0-1 Programming", Methods of Operations Research 12 (1971) 154-184. [13] Hadjiconstantinou, E., Lucas, C., Mitra, G., and Moody, S., "Tools for reformulating logical forms into zero-one mixed integer programs", European Journal of Operational Research 72 (1994) 263-277. [14] Hooker, J.N., "A quantitative approach to logical interference", Decision Support Systems 4 (1988) 45-69. [15] Hooker, J.N., "Solving the incremental satisfiability problem", Working Paper 1991-9, Graduate School of Industrial Administration, Carnegie-Mellon University, 1991. [16] Hooker, J.N, "Generalised resolution and cutting planes", Annals of Operations Research 12 (1988) 214-239. [17] Hooker, J.N., "Input Proofs and rank one cutting planes", ORSA Journal on Computing, to appear. [18] Hiirlimann, T., "IP, MIP and Logical Modelling using LPL", Paper presented at Symposium on Applied Mathematical Modelling and Optimisation, Budapest, January 1993. [19] Jeroslow, R.G., "Logic-based decision support", Annals of Discrete Mathematics 40 (1989). [20] Jeroslow, R.G., and Wang, J., "Dynamic Programming, integral polyhedra and Horn Clause knowledge bases", Working Paper, Georgia Institute of Technology, Atlanta, GA, 1987. [21] Langford, C.H., "Some theorems on deducibility", Annals of Mathematics 1 28 (1927) 16-40. [22] Loveland, D.W., Automated Theorem Proving: A Logical Basis, North-Holland, Amsterdam, 1978. [23] McKinnon, K.I.M., and Williams, H.P., "Constructing integer programming models by the predicate calculus", Annals of Operations Research' 21 (1989) 227-246. [24] Presburger, M., "Uber die Vollst5ndigkeit eines Gewissen Systems der Arithmetik ganzer Zahlen in Welchem die Addition als einzige Operation hervortritit", in: Comptes-Residus du I Congres de Mathematicians de pays Slaves, Warsaw, 1930, 92-101. [25] Quine, W.V., "The problem of simplifying truth functions", American Mathematical Monthly 59 (1952) 521-531. [26] Quine, W.V., "A way to simplify truth functions", Americal Mathematical Monthly 62 (1955) 627-631. [27] Robinson, J.A., "A machine-oriented logic based on the resolution principle", Journal of the ACM 12 (1965) 23-41. [28] Robinson, J.A., "A generalised resolution principle", Machine Intelligence 3 (1968) 77-93. [29] Wiliams, H.P., "An algorithm for the solution of linear programming problems", IBM Working Paper, 1970. [30] Williams, H.P., "Fourier-Motzkin elimination extension to integer programming problems", Journal of Combinatorial Theory 21 (1976) 118-123. [31] Williams, H.P., "Logical problems and integer programming", Bulletin of the IMA 13 (1977) 18-20. [32] Williams, H.P., "Fourier's method of linear programming and its dual", American Mathematical Monthly 93 (1986) 681-695. [33] Williams, H.P., "Linear and integer programming applied to the propositional calculus", International Journal of Systems Research and Information Science 2 (1987) 81-100. [34] Williams, H.P., "The elimination of integer variables", Journal of the Operations Research Society 43 (1992) 387-393. [35] Williams, H.P., "Computational logic and integer programming: Connections between the methods of logic, AI and OR", Paper presented at Symposium on Applied Mathematical Modelling and Optimisation, Brunel University, January 1991. [36] Wilson, J.M., "Methods of (0-1) integer programming using a Boolean Approach", D. Phil. Thesis, University of Sussex, 1975. [37] Wilson, J.M., "A method of (0-1) integer programming using a Boolean approach", in: T.B. Boffey (ed.) Proceedings of CP77 University of Liverpool, 1977.