- Email: [email protected]
Generating cuts in integer programming with families of special ordered sets
European Journal of Operational Research 46 (1990) 101-108 North-Holland
101
Theory and Methodology
Generating cuts in integer programming with families of special ordered sets J.M. W I L S O N
Department of Management Studies, Loughborough University of Technology, Loughborough, Leicestershire L E l l 3TU, England, UK
Abstract: This paper describes families of cuts which can be added to integer programming formulations incorporating special ordered sets. Ways in which the cuts can be used are discussed and some computational experience is presented.
Keywords: Mathematical programming, integer programming
1. Introduction
form
In integer programming formulations it is often necessary to model an event which takes place at one out of a sequence of possible times or an operation which is to take place at one of several positions. The time of the event or the position of the operation can then be determined. When two (or more) related events or operations are considered it is often possible to derive a series of inequalities, which act as cuts, which are satisfied because of exclusion or precedence requirements. Such relationships modelled by special ordered sets of variables were first discussed by Beale and Simons [2]. This paper will extend and develop the ideas of [2] and will start with a review of that work.
Ekhk, k
2. Mathematical background The time when an activity takes place or the position of the operation is computed by a term of
or more generally
ZfkXk k for constants fk where the Xk form a special ordered set of type one ($1) with a fixed upper bound of 1.0. An $1 set is an ordered set of nonnegative variables such that at most one may be nonzero. (For a discussion of $1 sets see [3,6]. Other related work on sets appear in [5,8, and 12].) Beale and Simons [2] consider occasions when the times of two activities or the positions of two operations are related by constraints of form
E k~.k -- E kP,k ~ 1 k k or, more generally E f * ~ k k -- E g k ~ k
k
Received March 1988; revised November 1988
k
~'~ C,
where the Xk and /~k both form S1 sets with fixed
0377-2217/90/$3.50 © 1990 - Elsevier Science Publishers B.V. (North-Holland)
J.M. Wilson / Generating cuts in integer programming
102
upper bounds of 1.0. Such $1 sets will be called precedence ordered sets (PS1).
In the second form of the constraint, c = 0 may be assumed without loss of generahty. Beale and Simons [2] show that for any real number X, such that fk ~ X, gk 4: X for any k, if S 1 denotes the set of values of k such that fk < X and S 2 the set of values of k such that gk > X, it then follows that
keS 1
keS 2
is a valid inequality which can act as a cut.
Example 1. Let if operation i is carried out at time k (i = 1 . . . . . 4; k = 1 , . . . , 7 ) ,
However, because of design features in the computer implementation of S1 sets, experience has shown that such inequalities lead to computational difficulties and should not be included explicitly. Satisfactory results can be obtained by use of equations Pi = E k ~ i k k
and associated relationships Pa
otherwise. Thus there are to be four S1 sets. In addition, suppose that operation i + 1 cannot be carried out until the time period after operation i (i = 1, 2, 3). Considering operations i and i + 1 we obtain the cuts j-1
7
E X,k + k= 1
E
X,+,,k>~l
(j=l
. . . . ,6).
k=j+l
More generally we also have cuts for nonadjacent operations, j--I
7
X,k+
X,+,,k>11
(•=2,3;
i+1~<4; j=l
. . . . . 6).
Such cuts will be termed paired precedence cuts. Note that if each S1 set is constrained to equal 1.0 the cuts can be represented as k~S2
which may be more compact. Note that it is not mandatory for the sets of )k's to be represented as S1 sets in a mathematical programming computer package. A set of binary variables which add to 1.0 will suffice. When the precedence relations of the example apply it is apparent that 4
Y'~ ~ik ~< 1 i~l
Xlk +
kES1
E
X2k + " ' " +
k~S z
E
2t,,k>/l"
(3.1)
k~S m
The inequality (3.1) will be termed a linked precedence cut. A generalisation of linked precedence cuts to activities described by Efk;kk
xkk~S~
Cuts are possible which join m a n y precedence relationships together. Let )ti, (i = 1 . . . . . m) be defined as in Example 1 and all its timing conditions apply. Consider integer numbers 1 ~< l~ < l 2 < • • • < 1,,_ 1 ~< n. Then if $1 denotes the set of values of k such that k < l~, $2 denotes the set of values of k such that l I < k < / 2 . . . . . and S m denotes the set of values of k such that l m _ 1 < k , then it follows that E
k=j+l
k=l
3. Linked precedence cuts
for each value of k.
k
is also possible. Generalisation to sets which have fixed upper bounds different from 1.0 will not be considered, following the lead of Beale and Simons [2]. In general it would be expected that linked precedence cuts have less power to aid convergence to a solution than paired precedence cuts.
Example 2. Using the data of Example 1 a typical linked precedence cut is 2t~1 + )k12 + ~24 + ~t25 + 7~37>~ 1.
J.M. Wilson / Generating cuts in integer programming
other combinations of this type. For some solutions the inequality will be nonbinding.
4. Vertical precedence cuts
So far cuts have been derived by looking at each S1 set in one direction which could be regarded as 'horizontal'. Earlier it was proposed that the 'vertical' relationship Ehi~< i
103
Example 3. Using the conditions of Example 2, a vertical precedence cut such as 3~k14 + 3~k24 q- ~k23 --}-~k22 q- ~k21
1
q- 2)k34 + )k33 + ~32 "t- ~31 q- ~k44 q- ~k43 q- X42 q- ~k4l ~ 3
should not be included explicitly (k = 1 . . . . . n). However, another vertical relationship is available. Let the precedence conditions developed in Section 3 continue to apply. The vertical precedence cut given by
(m-
i)Xik + ( m -
i)Xi+l, k + x i + L k _ 1+ . . .
q- ~i+1.1 + ( m - i - 1)~ki+2, k -4- ~ki+2,k_ 1 q- • • -
can be derived. As before, extensions are available for relationships of form
ELXk. k
A variety of families of vertical cuts can be derived.
-[- ~ki+ 2,1 + ( m -- i -- 2)~ki+3, k q- ~ki+3,k_ 1 q- "'"
+ Xi+3,1 + " " " +~krn,ik
+
~km,k-I +
"
"
"
q-~m,1
~<(m- i )
(4.1)
is a valid inequality. The proof is as follows.
Case 1. Xik = 1. Then because of precedence, Xyt = 0 if j > i and l ~< k. Hence all other variables in the inequality are zero and the inequality reduces to ( m - - i)Xik ~ ( m - - i). Case 2. Xi+l, k = 1. This case is similar to Case 1.
Case 3. X,k = 0 , Xi+l, k = 0 , and one of Xi+l,k_ 1, Xi+l,k_ =. . . . . Xi+I, 1 = 1. Then if: (a))',+Z,k = 1 precedence implies no other Xj, I with j > _ - i + 2 , l ~ k , can be nonzero and the inequality reduces to l + (rn-i-1)
5. Exclusive sets and cuts
When there are several $1 sets there may be an exclusive relationship to be described by them, e.g.: Xk = 1 if operation 1 is carried out at time k (k=l
.... ,n),
/~k = 1 if operation 2 is carried out at time k (k = 1 . . . . . n), and operations 1 and 2 are to be carried out at different times. Such $1 sets will be called exclusive ordered sets ES1. Thus the relationships ~kk q- P'k ~ ]
are implicit, but as stated earlier it is not advisable for them to be included if the X,'s and ~t,'s are to form $1 sets. An alternative is to include the pair of inequalities
<~( m - i ) .
(b) X i + : a = l , 1 4 = k - 1 , and X i + 3 . k = I , then precedence implies no other Xjt with j > i + 3 and 1 ~< k can be nonzero. The inequality reduces to
l + l +(m-i-2)<~(m-i). (c) Xi+z,t= 1, 14=k, and Xi+3,1= 1, 14=k, and X,+4, k = 1, then precedence implies no other Xj/ with j > t i + 4 and l~
1 + E k~kk ~ kl~k + nS, k=l k=l
i kXk> I2 "
k=l
(5.1)
(5.2)
k=l
where 3 ~ (0, 1). These will prove effective as cuts if branching on 8 is given high priority. Such constraints have been called "disjunctive" and a discussion of their formulation appears in Jeroslow and Lowe [7]. Cuts such as
l + l + l + (m-i-3)~(m-i).
ga,+
Y'#i>~l
The process of (a)-(c) can now be continued for
i#=j
i~j
for some j ~ (1 . . . . . n}
104
J.M. Wilson / Generating cuts in mteger programming
are possible and their generalisations to more than two S1 sets,
creasing in k and X, k~,
k 2 are
chosen such that
f*,-a < X < f k , , Y] %ii > ~ n - [ J I
(i=1 ..... m) gk2 < X <
j~J
for some J _c (1 . . . . . n },
(5.3)
are available. Cuts such as (5.3) will be called exclusive cuts. Example 4. Suppose the continuous optimum has provided the values %1 = th = 0.25, %2 =/~2 = 0.75 where m = 2 and n = 3 , then 0.33~<8~<0.67 satisfies (5.1) and (5.2) but the exclusive cut %1 + ~ l " b ~ ' 3 q - P 3
>~1
is not satisfied. The strength of such cuts may be limited and neither exclusive cuts of the above form nor branching on 8 can prevent solutions such as %1 = %2 = l-t2 = ~3 = 0.5 from arising. Related problems involving constraints more general than (5.1) and (5.2) of form - M S + ~'~ ak% k ~< b, k
where M is a large constant are discussed in Johnson et al. [8]. These problems do not allow exclusive cuts, but some other cuts are possible.
6. Extensions to special ordered sets of type 2 ($2) An $2 set is an ordered set of nonnegative variables such that at most two may be nonzero and if two are nonzero they must be adjacent in the order. For paired precedence cuts, Beale and Simons [2] show that the cuts generalise to $2 sets with fixed upper bounds of 1.0. The inequalities hold but the sets S~ and $2 have to be modified to ensure that for all k~ ~ S~ and k 2 ~ S 2 for f and g defined as in Section 2,
gk2+l ;
the cut is E
%*+
k <~k1
E
/xk>~l
k >k2
for the cuts of Section 2. In fact this cut from [2] is rather weaker than it need be and if X, k 1, k 2 are chosen such that
fk,
E ~kkq- ~" IXk~1 k<~k 1
k>~k 2
is valid. The linked precedence cuts of Section 3 are valid for $2 sets if S a is chosen for k <~Il, S z is chosen for /1 ~< k ~< l 2 . . . . . and S,, is chosen for l,,_ a ~ k. The vertical precedence cuts of Section 4 have a simple extension to $2 sets. A generalisation of ES1 to $2 sets is possible. It will be assumed that the exclusive relationship requires that %k ~/Xk (k = 1 . . . . . n) for nonzero values and then ES2 sets can be defined by the inequalities (5.1) and (5.2) and the relationships %k+/tk~l remain implicit.
7. Chains of linked ordered sets Chains of linked ordered sets were introduced by Beale and Forrest [1]. For the $2 case, a series of linked sets could be formulated as
fk, - 1 - g*2 > 0, fk,+l--gk2 > 0 ,
fk,-
g,2-1 > 0,
fk, - gk2+ 1 > 0, and Beale and Simons [2] show that for the particular case where fk and gk are monotonic in-
Y'. % i k = l
(i=1 ..... m)
k~K
in which %~k =~ 0 ~ %tk =~ 0 (l = 1 . . . . . m), and each {%~k: i = 1 . . . . . m; k ~ K } is an $2 set. Generalisations exist to sets with an upper bound different from 1, but there is not usually a need for sets of S1 type.
J.M. Wilson / Generating cuts in integer programming
Again cuts may be derived for such sets as follows: The cut ~ i l , j -~- ~i2,j+2 "1- ~ki3,j+4 ~- " " " ~ 1, il, i 2 ....
~(1
.....
m),
il--/=i 2 . . . . .
is a valid cut for linked ordered sets (LS2).
Example 5. If h 1 + ~k2 h- ~k3 "+- ~k4 "q- ~k5 q- ~k6 = 1,
105
plete independent r o l e - - f o r instance to model multiple choice relationships. However, when sets are included as a family there are three likely purposes: (a) The nonzeroes in the sets must form a precedence of columns - - P S 1 sets. (b) The nonzeroes in the sets must not overlap in c o l u m n s - - E S 1 sets. (c) The nonzeroes must all occur in the same c o l u m n s - - LS2 sets.
and
9. Using the cuts define LS2 sets. Then the trial solution ~2 = 0.25,
?~3 = 0.75,
]~4 = 0.25,
/z 5 = 0.75,
which may arise from a continuous optimum can be barred by the cut
The mathematical programming system SCIC O N I C / V M [10] has special facilities for the incorporation of LS2 sets. Such facilities may not be available in other systems. However, the LS2 relationships may be formulated by other sets and variables as follows. To define k linked ordered sets, introduce the variables 81. . . . . 8n such that each 8 is bounded by zero and one, and E~= l~j is an $2 set. Then the $2 sets n
E x,j=l
(/=1 ..... k)
j=l
together with the relationships k E ?~,j=kSj i=1
for each j ~
{1 . . . . . n}
suffice. Cuts may now be incorporated with the above relationships.
8. A family of sets In the preceding sections different groupings of S1 or $2 sets have been discussed. Such sets occur when formulations make use of families of sets to describe relationships. Many S1 sets may occur in a formulation with each one performing a c o m -
The cuts can be added to an integer programming formulation to attempt to narrow the gap between a continuous o p t i m u m and a feasible integer solution at any stage in the branch and bound process. As the branch and bound algorithm progresses and more variables take restricted values, the cuts become more effective. The difficulty remains of knowing which selection of cuts to add to the formulation as adding all possible cuts would result in an unwieldy continuous problem. Crowder et al. [4] describe procedures for solving integer programming problems which involve the generation of additional constraints 'on the fly' as the branch and bound algorithm progresses. Their work does not fully exploit S1 and $2 set properties. Thus there would appear to be scope for adding the cuts for S1 and $2 sets as the solution process progresses. Crowder et al. [4] suggest as a procedure that additional contraints should be added until the gain in the objective function value becomes less than some arbitrary limit. When this improvement criterion is not met, the normal branch and bound algorithm continues. It is possible to add cuts after any branch and doing this can help stop the tendency of one high priority variable which took an integer value by chance from moving away from this value as branching takes place on other variables and then being forced to branch to a value other than that which it had earlier.
10. Computational results To examine paired precedence cuts, a test problem was constructed with 10 S1 sets each of 20
J.hI. Wilson / Generating cuts in integer programming
106 Table 1 Problem with cuts added successively Problem
P0
P1
P2
P3
P4
P5
P6
P7
P8
P9
Nodes Iterations CPUsecs
177 11,16 262
129 857 182
175 1335 324
89 930 256
95 1536 375
71 1549 430
17 896 306
9 670 237
3 416 172
1 216 121
variables, where 20
Y~)~,j=I
( i = 1 . . . . . 10),
j=l 20
P,= EJ~,s
( i = l . . . . . 10),
j=l
Pi
(i =
1 . . . . . 9),
and every ~,j appears in the objective function, which is to be minimised, with a coefficient chosen at random from the range 1.0-10.0. The problem was run without cuts (P0) to optimality and then with successively adding the 18 paired precedence cuts between S1 sets 1 and 2 (problem P1), then the 18 paired precedence cuts between S1 and 3 (problem P2) and so on creating problems P0-P9. Results are given in Table 1. The results in Table 1 were obtained using an integer programming package built from MINOS [9] and run times are CPU seconds on a VAX. The table shows that the first few sets of cuts make some improvements to solution time to optimality. Thereafter CPU times rise as more cuts are added and then times drop dramatically as the final sets of cuts are added finally reducing CPU time to less than half that of the formulation without cuts. As might be expected, the time taken for each simplex iteration rises as cuts are added. (The line "Iterations" in Table 1 gives the number of simplex iterations used.) The progressive addition of
cuts leads to a larger problem size and corresponding greater overhead for the simplex algorithm. The problem was now return with no paired precedence cuts added but with vertical precedence cuts added progressively. Table 2 illustrates the results of adding a selection of cuts progressively linking columns in ascending order. The results of Table 2 show that it appears to be worthwhile adding a small number of cuts on the first few columns of the problem. Thereafter the problem becomes cumbersome and although the number of branch and bound nodes becomes reduced, ultimately CPU times rise. Further computational work on larger problems was then conducted using S C I C O N I C / V M [10] on a Prime 750 computer. A series of problems was generated and solved in three different ways. Table 3 shows the results of solving the problems when no cuts are added. Table 4 shows the results of solving the problems when all possible paired precedence cuts are added. Finally, Table 5 shows the results of solving the problems with five vertical precedence cuts added to each original problem. This number of cuts was found to be more successful than any smaller number. For all but two of the problems, Table 4 shows good reductions in computational time required. For all problems, few branch and bound nodes are
Table 2 Problem with vertical precedence cuts added progressively Cut Nodes Iterations CPUsecs
0 177 1146 262
1 133 1165 200
2 163 1366 240
3 111 1019 196
4 169 1508 277
5 165 1578 280
6 165 1917 345
Cut Nodes Iterations CPUsecs
10 165 2208 452
11 167 2637 464
12 61 1398 255
13 113 2248 396
14 80 1702 308
15 84 1730 322
16 74 1797 322
7 169 1816 324
8 111 1903 320
9 103 1994 327
J.M. Wilson / Generating cuts in integer programming
107
Table 3 Problems solved without cuts Problem
Size mx n
Total CPU to complete search
Iterations to optimal
Nodes to optimal
Total nodes
No. of sets
1 2 3 4 5 6 7 8 9
22 × 175 32 x 225 22 × 225 32 x 300 42 x 350 22 x 275 32 x 375 42 x 450 52 × 500
188 144 64 780 660 46 1261 4289 2610
416 182 160 246 633 86 1559 3839 2063
324 101 92 164 551 35 1127 3205 1338
465 231 118 1117 733 71 1585 4346 2094
10 15 10 15 20 10 15 20 25
required. In the unsuccessful problems, numbered 3 and 6, the difference between the number of variables in each S1 set and the number of sets is at its highest for all the problems. Again in Table 5 these two problems are seen to perform badly. Of the remaining problems, those numbered 1, 2,
Table 4 Problems solved with paired precedence cuts Problem Size m× n
1 2 3 4 5 6 7 8 9
Iterations Nodes Total CPU to to optimal to optimal complete search
148×175 128 193 × 225 85 193×225 112 263×300 250 308x350 244 238 x 275 61 333x375 427 403x450 645 448 x 500 1194
375 336 305 555 393 157 526 495 832
13 4 3 3 2 1 3 1 5
Total nodes
13 5 5 3 3 1 5 1 7
Table 5 Problems solved with vertical precedence cuts
4, 5 and 7 perform better with the vertical cuts as shown in Table 5 than in Table 3 but not as well as in Table 4. Problems 8 and 9, the largest problems, perform worse as shown in Table 5 than in Table 3. Thus the evidence seems to suggest that the paired precedence cuts are the more useful, but should not be used when the number of variables in each $1 set is much larger than the number of sets. Simons [11] discusses using paired precedence cuts on a large maintenance problem and finds it to be more effective than using techniques suggested by Van Roy and Wolsey [13] for adding cuts to the problem. Both these sets of experiment were carried out using the S C I C O N I C / V M [10]. It remains open as to whether generating cuts during the optimization process, especially violated cuts or cutting planes, would also lead to further improvements in computation time. However, S C I C O N I C / V M [10] is not a convenient piece of software for such experimentation and it was not attempted by this author.
References
Problem
Size m× n
Total CPU to complete search
Iterations to optimal
Total nodes
Nodes to optimal
1 2 3 4 5 6 7 8 9
27 × 175 37 x 225 27 × 225 37 × 300 47 × 350 27 x 275 37 × 375 47 ×450 57×500
134 125 94 752 610 79 919 4781 2621
383 328 201 753 1127 172 969 2033 3143
266 161 171 975 575 124 1028 4284 1884
177 130 124 398 504 113 420 3002 1692
[1] Beale, E.M.L., and Forrest, J.J.H., "Global optimization as an extension of integer programming", in: L.C.W. Dixon and G.P. Szego (eds.), Towards Global Optimization 2, North-Holland, Amsterdam, 1978, 131-149. [2] Beale, E.M.L., and Simons, R.V., "Generating cuts from precedence constraints and some generalisations", Research report, Scicon Computing Services, Milton Keynes, 1986. [3] Beale, E.M.L, and Tomlin, J.A., "Special facilities in a general programming system for nonconvex problems using ordered sets of variables", in: J. Lawrence (ed.),
Proceedings of the 5th International Conference of O.R, Tavistock, London, 1970, 447-454.
108
J.M. Wilson / Generating cuts in integer programming
[4] Crowder, H. Johnson, E.L., and Padberg, M.W., "Solving large scale zero-one finear programming problems", Operations Research 31 (1983) 803-834. [5] Escudero, L.F., "$3 sets. An extension of the BealeTomlin special ordered sets", Mathematical Programming 42 (1988) 113-123. [6] Hummeltenberg, W., "Implementation of special ordered sets in MP software", European Journal of Operational Research 17 (1983) 1-15. [7] Jeroslow, R.G., and Lowe, J.K., "Experimental results with the new techniques for integer programming formulation", Journal of the Operational Research Society 36 (1985) 393-403. [8] Johnson, E.L., Kostreva, M.L., and Suhl, U.H., "Solving 0-1 integer programming problems arising from large scale planning models", Operations Research 33 (1985) 803-819.
[9] Murtagh, B.A., and Saunders, M.A., "Minos 5.0 users guide", Report SOL83-20, Department of Operations Research, Stanford University, CA, 1983. [10] Sciconic/VM User Guide, Scicon Computing Services, Milton Keynes, 1987. [1l] Simons, R.V., "Some attempts at applying integer programming with cuts to practical combinatorial problems", Paper presented at CO87, University of Southampton, August 1987. [12] Snyder, R.D., "Linear programming with special ordered sets", Journal of the Operational Research Society 35 (1984) 69-74. [13] Van Roy, T.J., and Wolsey, L.A., "Solving mixed integer programming problems using automatic reformulation", Operations Research 35 (1987) 45-57.
101
Theory and Methodology
Generating cuts in integer programming with families of special ordered sets J.M. W I L S O N
Department of Management Studies, Loughborough University of Technology, Loughborough, Leicestershire L E l l 3TU, England, UK
Abstract: This paper describes families of cuts which can be added to integer programming formulations incorporating special ordered sets. Ways in which the cuts can be used are discussed and some computational experience is presented.
Keywords: Mathematical programming, integer programming
1. Introduction
form
In integer programming formulations it is often necessary to model an event which takes place at one out of a sequence of possible times or an operation which is to take place at one of several positions. The time of the event or the position of the operation can then be determined. When two (or more) related events or operations are considered it is often possible to derive a series of inequalities, which act as cuts, which are satisfied because of exclusion or precedence requirements. Such relationships modelled by special ordered sets of variables were first discussed by Beale and Simons [2]. This paper will extend and develop the ideas of [2] and will start with a review of that work.
Ekhk, k
2. Mathematical background The time when an activity takes place or the position of the operation is computed by a term of
or more generally
ZfkXk k for constants fk where the Xk form a special ordered set of type one ($1) with a fixed upper bound of 1.0. An $1 set is an ordered set of nonnegative variables such that at most one may be nonzero. (For a discussion of $1 sets see [3,6]. Other related work on sets appear in [5,8, and 12].) Beale and Simons [2] consider occasions when the times of two activities or the positions of two operations are related by constraints of form
E k~.k -- E kP,k ~ 1 k k or, more generally E f * ~ k k -- E g k ~ k
k
Received March 1988; revised November 1988
k
~'~ C,
where the Xk and /~k both form S1 sets with fixed
0377-2217/90/$3.50 © 1990 - Elsevier Science Publishers B.V. (North-Holland)
J.M. Wilson / Generating cuts in integer programming
102
upper bounds of 1.0. Such $1 sets will be called precedence ordered sets (PS1).
In the second form of the constraint, c = 0 may be assumed without loss of generahty. Beale and Simons [2] show that for any real number X, such that fk ~ X, gk 4: X for any k, if S 1 denotes the set of values of k such that fk < X and S 2 the set of values of k such that gk > X, it then follows that
keS 1
keS 2
is a valid inequality which can act as a cut.
Example 1. Let if operation i is carried out at time k (i = 1 . . . . . 4; k = 1 , . . . , 7 ) ,
However, because of design features in the computer implementation of S1 sets, experience has shown that such inequalities lead to computational difficulties and should not be included explicitly. Satisfactory results can be obtained by use of equations Pi = E k ~ i k k
and associated relationships Pa
otherwise. Thus there are to be four S1 sets. In addition, suppose that operation i + 1 cannot be carried out until the time period after operation i (i = 1, 2, 3). Considering operations i and i + 1 we obtain the cuts j-1
7
E X,k + k= 1
E
X,+,,k>~l
(j=l
. . . . ,6).
k=j+l
More generally we also have cuts for nonadjacent operations, j--I
7
X,k+
X,+,,k>11
(•=2,3;
i+1~<4; j=l
. . . . . 6).
Such cuts will be termed paired precedence cuts. Note that if each S1 set is constrained to equal 1.0 the cuts can be represented as k~S2
which may be more compact. Note that it is not mandatory for the sets of )k's to be represented as S1 sets in a mathematical programming computer package. A set of binary variables which add to 1.0 will suffice. When the precedence relations of the example apply it is apparent that 4
Y'~ ~ik ~< 1 i~l
Xlk +
kES1
E
X2k + " ' " +
k~S z
E
2t,,k>/l"
(3.1)
k~S m
The inequality (3.1) will be termed a linked precedence cut. A generalisation of linked precedence cuts to activities described by Efk;kk
xkk~S~
Cuts are possible which join m a n y precedence relationships together. Let )ti, (i = 1 . . . . . m) be defined as in Example 1 and all its timing conditions apply. Consider integer numbers 1 ~< l~ < l 2 < • • • < 1,,_ 1 ~< n. Then if $1 denotes the set of values of k such that k < l~, $2 denotes the set of values of k such that l I < k < / 2 . . . . . and S m denotes the set of values of k such that l m _ 1 < k , then it follows that E
k=j+l
k=l
3. Linked precedence cuts
for each value of k.
k
is also possible. Generalisation to sets which have fixed upper bounds different from 1.0 will not be considered, following the lead of Beale and Simons [2]. In general it would be expected that linked precedence cuts have less power to aid convergence to a solution than paired precedence cuts.
Example 2. Using the data of Example 1 a typical linked precedence cut is 2t~1 + )k12 + ~24 + ~t25 + 7~37>~ 1.
J.M. Wilson / Generating cuts in integer programming
other combinations of this type. For some solutions the inequality will be nonbinding.
4. Vertical precedence cuts
So far cuts have been derived by looking at each S1 set in one direction which could be regarded as 'horizontal'. Earlier it was proposed that the 'vertical' relationship Ehi~< i
103
Example 3. Using the conditions of Example 2, a vertical precedence cut such as 3~k14 + 3~k24 q- ~k23 --}-~k22 q- ~k21
1
q- 2)k34 + )k33 + ~32 "t- ~31 q- ~k44 q- ~k43 q- X42 q- ~k4l ~ 3
should not be included explicitly (k = 1 . . . . . n). However, another vertical relationship is available. Let the precedence conditions developed in Section 3 continue to apply. The vertical precedence cut given by
(m-
i)Xik + ( m -
i)Xi+l, k + x i + L k _ 1+ . . .
q- ~i+1.1 + ( m - i - 1)~ki+2, k -4- ~ki+2,k_ 1 q- • • -
can be derived. As before, extensions are available for relationships of form
ELXk. k
A variety of families of vertical cuts can be derived.
-[- ~ki+ 2,1 + ( m -- i -- 2)~ki+3, k q- ~ki+3,k_ 1 q- "'"
+ Xi+3,1 + " " " +~krn,ik
+
~km,k-I +
"
"
"
q-~m,1
~<(m- i )
(4.1)
is a valid inequality. The proof is as follows.
Case 1. Xik = 1. Then because of precedence, Xyt = 0 if j > i and l ~< k. Hence all other variables in the inequality are zero and the inequality reduces to ( m - - i)Xik ~ ( m - - i). Case 2. Xi+l, k = 1. This case is similar to Case 1.
Case 3. X,k = 0 , Xi+l, k = 0 , and one of Xi+l,k_ 1, Xi+l,k_ =. . . . . Xi+I, 1 = 1. Then if: (a))',+Z,k = 1 precedence implies no other Xj, I with j > _ - i + 2 , l ~ k , can be nonzero and the inequality reduces to l + (rn-i-1)
5. Exclusive sets and cuts
When there are several $1 sets there may be an exclusive relationship to be described by them, e.g.: Xk = 1 if operation 1 is carried out at time k (k=l
.... ,n),
/~k = 1 if operation 2 is carried out at time k (k = 1 . . . . . n), and operations 1 and 2 are to be carried out at different times. Such $1 sets will be called exclusive ordered sets ES1. Thus the relationships ~kk q- P'k ~ ]
are implicit, but as stated earlier it is not advisable for them to be included if the X,'s and ~t,'s are to form $1 sets. An alternative is to include the pair of inequalities
<~( m - i ) .
(b) X i + : a = l , 1 4 = k - 1 , and X i + 3 . k = I , then precedence implies no other Xjt with j > i + 3 and 1 ~< k can be nonzero. The inequality reduces to
l + l +(m-i-2)<~(m-i). (c) Xi+z,t= 1, 14=k, and Xi+3,1= 1, 14=k, and X,+4, k = 1, then precedence implies no other Xj/ with j > t i + 4 and l~
1 + E k~kk ~ kl~k + nS, k=l k=l
i kXk> I2 "
k=l
(5.1)
(5.2)
k=l
where 3 ~ (0, 1). These will prove effective as cuts if branching on 8 is given high priority. Such constraints have been called "disjunctive" and a discussion of their formulation appears in Jeroslow and Lowe [7]. Cuts such as
l + l + l + (m-i-3)~(m-i).
ga,+
Y'#i>~l
The process of (a)-(c) can now be continued for
i#=j
i~j
for some j ~ (1 . . . . . n}
104
J.M. Wilson / Generating cuts in mteger programming
are possible and their generalisations to more than two S1 sets,
creasing in k and X, k~,
k 2 are
chosen such that
f*,-a < X < f k , , Y] %ii > ~ n - [ J I
(i=1 ..... m) gk2 < X <
j~J
for some J _c (1 . . . . . n },
(5.3)
are available. Cuts such as (5.3) will be called exclusive cuts. Example 4. Suppose the continuous optimum has provided the values %1 = th = 0.25, %2 =/~2 = 0.75 where m = 2 and n = 3 , then 0.33~<8~<0.67 satisfies (5.1) and (5.2) but the exclusive cut %1 + ~ l " b ~ ' 3 q - P 3
>~1
is not satisfied. The strength of such cuts may be limited and neither exclusive cuts of the above form nor branching on 8 can prevent solutions such as %1 = %2 = l-t2 = ~3 = 0.5 from arising. Related problems involving constraints more general than (5.1) and (5.2) of form - M S + ~'~ ak% k ~< b, k
where M is a large constant are discussed in Johnson et al. [8]. These problems do not allow exclusive cuts, but some other cuts are possible.
6. Extensions to special ordered sets of type 2 ($2) An $2 set is an ordered set of nonnegative variables such that at most two may be nonzero and if two are nonzero they must be adjacent in the order. For paired precedence cuts, Beale and Simons [2] show that the cuts generalise to $2 sets with fixed upper bounds of 1.0. The inequalities hold but the sets S~ and $2 have to be modified to ensure that for all k~ ~ S~ and k 2 ~ S 2 for f and g defined as in Section 2,
gk2+l ;
the cut is E
%*+
k <~k1
E
/xk>~l
k >k2
for the cuts of Section 2. In fact this cut from [2] is rather weaker than it need be and if X, k 1, k 2 are chosen such that
fk,
E ~kkq- ~" IXk~1 k<~k 1
k>~k 2
is valid. The linked precedence cuts of Section 3 are valid for $2 sets if S a is chosen for k <~Il, S z is chosen for /1 ~< k ~< l 2 . . . . . and S,, is chosen for l,,_ a ~ k. The vertical precedence cuts of Section 4 have a simple extension to $2 sets. A generalisation of ES1 to $2 sets is possible. It will be assumed that the exclusive relationship requires that %k ~/Xk (k = 1 . . . . . n) for nonzero values and then ES2 sets can be defined by the inequalities (5.1) and (5.2) and the relationships %k+/tk~l remain implicit.
7. Chains of linked ordered sets Chains of linked ordered sets were introduced by Beale and Forrest [1]. For the $2 case, a series of linked sets could be formulated as
fk, - 1 - g*2 > 0, fk,+l--gk2 > 0 ,
fk,-
g,2-1 > 0,
fk, - gk2+ 1 > 0, and Beale and Simons [2] show that for the particular case where fk and gk are monotonic in-
Y'. % i k = l
(i=1 ..... m)
k~K
in which %~k =~ 0 ~ %tk =~ 0 (l = 1 . . . . . m), and each {%~k: i = 1 . . . . . m; k ~ K } is an $2 set. Generalisations exist to sets with an upper bound different from 1, but there is not usually a need for sets of S1 type.
J.M. Wilson / Generating cuts in integer programming
Again cuts may be derived for such sets as follows: The cut ~ i l , j -~- ~i2,j+2 "1- ~ki3,j+4 ~- " " " ~ 1, il, i 2 ....
~(1
.....
m),
il--/=i 2 . . . . .
is a valid cut for linked ordered sets (LS2).
Example 5. If h 1 + ~k2 h- ~k3 "+- ~k4 "q- ~k5 q- ~k6 = 1,
105
plete independent r o l e - - f o r instance to model multiple choice relationships. However, when sets are included as a family there are three likely purposes: (a) The nonzeroes in the sets must form a precedence of columns - - P S 1 sets. (b) The nonzeroes in the sets must not overlap in c o l u m n s - - E S 1 sets. (c) The nonzeroes must all occur in the same c o l u m n s - - LS2 sets.
and
9. Using the cuts define LS2 sets. Then the trial solution ~2 = 0.25,
?~3 = 0.75,
]~4 = 0.25,
/z 5 = 0.75,
which may arise from a continuous optimum can be barred by the cut
The mathematical programming system SCIC O N I C / V M [10] has special facilities for the incorporation of LS2 sets. Such facilities may not be available in other systems. However, the LS2 relationships may be formulated by other sets and variables as follows. To define k linked ordered sets, introduce the variables 81. . . . . 8n such that each 8 is bounded by zero and one, and E~= l~j is an $2 set. Then the $2 sets n
E x,j=l
(/=1 ..... k)
j=l
together with the relationships k E ?~,j=kSj i=1
for each j ~
{1 . . . . . n}
suffice. Cuts may now be incorporated with the above relationships.
8. A family of sets In the preceding sections different groupings of S1 or $2 sets have been discussed. Such sets occur when formulations make use of families of sets to describe relationships. Many S1 sets may occur in a formulation with each one performing a c o m -
The cuts can be added to an integer programming formulation to attempt to narrow the gap between a continuous o p t i m u m and a feasible integer solution at any stage in the branch and bound process. As the branch and bound algorithm progresses and more variables take restricted values, the cuts become more effective. The difficulty remains of knowing which selection of cuts to add to the formulation as adding all possible cuts would result in an unwieldy continuous problem. Crowder et al. [4] describe procedures for solving integer programming problems which involve the generation of additional constraints 'on the fly' as the branch and bound algorithm progresses. Their work does not fully exploit S1 and $2 set properties. Thus there would appear to be scope for adding the cuts for S1 and $2 sets as the solution process progresses. Crowder et al. [4] suggest as a procedure that additional contraints should be added until the gain in the objective function value becomes less than some arbitrary limit. When this improvement criterion is not met, the normal branch and bound algorithm continues. It is possible to add cuts after any branch and doing this can help stop the tendency of one high priority variable which took an integer value by chance from moving away from this value as branching takes place on other variables and then being forced to branch to a value other than that which it had earlier.
10. Computational results To examine paired precedence cuts, a test problem was constructed with 10 S1 sets each of 20
J.hI. Wilson / Generating cuts in integer programming
106 Table 1 Problem with cuts added successively Problem
P0
P1
P2
P3
P4
P5
P6
P7
P8
P9
Nodes Iterations CPUsecs
177 11,16 262
129 857 182
175 1335 324
89 930 256
95 1536 375
71 1549 430
17 896 306
9 670 237
3 416 172
1 216 121
variables, where 20
Y~)~,j=I
( i = 1 . . . . . 10),
j=l 20
P,= EJ~,s
( i = l . . . . . 10),
j=l
Pi
(i =
1 . . . . . 9),
and every ~,j appears in the objective function, which is to be minimised, with a coefficient chosen at random from the range 1.0-10.0. The problem was run without cuts (P0) to optimality and then with successively adding the 18 paired precedence cuts between S1 sets 1 and 2 (problem P1), then the 18 paired precedence cuts between S1 and 3 (problem P2) and so on creating problems P0-P9. Results are given in Table 1. The results in Table 1 were obtained using an integer programming package built from MINOS [9] and run times are CPU seconds on a VAX. The table shows that the first few sets of cuts make some improvements to solution time to optimality. Thereafter CPU times rise as more cuts are added and then times drop dramatically as the final sets of cuts are added finally reducing CPU time to less than half that of the formulation without cuts. As might be expected, the time taken for each simplex iteration rises as cuts are added. (The line "Iterations" in Table 1 gives the number of simplex iterations used.) The progressive addition of
cuts leads to a larger problem size and corresponding greater overhead for the simplex algorithm. The problem was now return with no paired precedence cuts added but with vertical precedence cuts added progressively. Table 2 illustrates the results of adding a selection of cuts progressively linking columns in ascending order. The results of Table 2 show that it appears to be worthwhile adding a small number of cuts on the first few columns of the problem. Thereafter the problem becomes cumbersome and although the number of branch and bound nodes becomes reduced, ultimately CPU times rise. Further computational work on larger problems was then conducted using S C I C O N I C / V M [10] on a Prime 750 computer. A series of problems was generated and solved in three different ways. Table 3 shows the results of solving the problems when no cuts are added. Table 4 shows the results of solving the problems when all possible paired precedence cuts are added. Finally, Table 5 shows the results of solving the problems with five vertical precedence cuts added to each original problem. This number of cuts was found to be more successful than any smaller number. For all but two of the problems, Table 4 shows good reductions in computational time required. For all problems, few branch and bound nodes are
Table 2 Problem with vertical precedence cuts added progressively Cut Nodes Iterations CPUsecs
0 177 1146 262
1 133 1165 200
2 163 1366 240
3 111 1019 196
4 169 1508 277
5 165 1578 280
6 165 1917 345
Cut Nodes Iterations CPUsecs
10 165 2208 452
11 167 2637 464
12 61 1398 255
13 113 2248 396
14 80 1702 308
15 84 1730 322
16 74 1797 322
7 169 1816 324
8 111 1903 320
9 103 1994 327
J.M. Wilson / Generating cuts in integer programming
107
Table 3 Problems solved without cuts Problem
Size mx n
Total CPU to complete search
Iterations to optimal
Nodes to optimal
Total nodes
No. of sets
1 2 3 4 5 6 7 8 9
22 × 175 32 x 225 22 × 225 32 x 300 42 x 350 22 x 275 32 x 375 42 x 450 52 × 500
188 144 64 780 660 46 1261 4289 2610
416 182 160 246 633 86 1559 3839 2063
324 101 92 164 551 35 1127 3205 1338
465 231 118 1117 733 71 1585 4346 2094
10 15 10 15 20 10 15 20 25
required. In the unsuccessful problems, numbered 3 and 6, the difference between the number of variables in each S1 set and the number of sets is at its highest for all the problems. Again in Table 5 these two problems are seen to perform badly. Of the remaining problems, those numbered 1, 2,
Table 4 Problems solved with paired precedence cuts Problem Size m× n
1 2 3 4 5 6 7 8 9
Iterations Nodes Total CPU to to optimal to optimal complete search
148×175 128 193 × 225 85 193×225 112 263×300 250 308x350 244 238 x 275 61 333x375 427 403x450 645 448 x 500 1194
375 336 305 555 393 157 526 495 832
13 4 3 3 2 1 3 1 5
Total nodes
13 5 5 3 3 1 5 1 7
Table 5 Problems solved with vertical precedence cuts
4, 5 and 7 perform better with the vertical cuts as shown in Table 5 than in Table 3 but not as well as in Table 4. Problems 8 and 9, the largest problems, perform worse as shown in Table 5 than in Table 3. Thus the evidence seems to suggest that the paired precedence cuts are the more useful, but should not be used when the number of variables in each $1 set is much larger than the number of sets. Simons [11] discusses using paired precedence cuts on a large maintenance problem and finds it to be more effective than using techniques suggested by Van Roy and Wolsey [13] for adding cuts to the problem. Both these sets of experiment were carried out using the S C I C O N I C / V M [10]. It remains open as to whether generating cuts during the optimization process, especially violated cuts or cutting planes, would also lead to further improvements in computation time. However, S C I C O N I C / V M [10] is not a convenient piece of software for such experimentation and it was not attempted by this author.
References
Problem
Size m× n
Total CPU to complete search
Iterations to optimal
Total nodes
Nodes to optimal
1 2 3 4 5 6 7 8 9
27 × 175 37 x 225 27 × 225 37 × 300 47 × 350 27 x 275 37 × 375 47 ×450 57×500
134 125 94 752 610 79 919 4781 2621
383 328 201 753 1127 172 969 2033 3143
266 161 171 975 575 124 1028 4284 1884
177 130 124 398 504 113 420 3002 1692
[1] Beale, E.M.L., and Forrest, J.J.H., "Global optimization as an extension of integer programming", in: L.C.W. Dixon and G.P. Szego (eds.), Towards Global Optimization 2, North-Holland, Amsterdam, 1978, 131-149. [2] Beale, E.M.L., and Simons, R.V., "Generating cuts from precedence constraints and some generalisations", Research report, Scicon Computing Services, Milton Keynes, 1986. [3] Beale, E.M.L, and Tomlin, J.A., "Special facilities in a general programming system for nonconvex problems using ordered sets of variables", in: J. Lawrence (ed.),
Proceedings of the 5th International Conference of O.R, Tavistock, London, 1970, 447-454.
108
J.M. Wilson / Generating cuts in integer programming
[4] Crowder, H. Johnson, E.L., and Padberg, M.W., "Solving large scale zero-one finear programming problems", Operations Research 31 (1983) 803-834. [5] Escudero, L.F., "$3 sets. An extension of the BealeTomlin special ordered sets", Mathematical Programming 42 (1988) 113-123. [6] Hummeltenberg, W., "Implementation of special ordered sets in MP software", European Journal of Operational Research 17 (1983) 1-15. [7] Jeroslow, R.G., and Lowe, J.K., "Experimental results with the new techniques for integer programming formulation", Journal of the Operational Research Society 36 (1985) 393-403. [8] Johnson, E.L., Kostreva, M.L., and Suhl, U.H., "Solving 0-1 integer programming problems arising from large scale planning models", Operations Research 33 (1985) 803-819.
[9] Murtagh, B.A., and Saunders, M.A., "Minos 5.0 users guide", Report SOL83-20, Department of Operations Research, Stanford University, CA, 1983. [10] Sciconic/VM User Guide, Scicon Computing Services, Milton Keynes, 1987. [1l] Simons, R.V., "Some attempts at applying integer programming with cuts to practical combinatorial problems", Paper presented at CO87, University of Southampton, August 1987. [12] Snyder, R.D., "Linear programming with special ordered sets", Journal of the Operational Research Society 35 (1984) 69-74. [13] Van Roy, T.J., and Wolsey, L.A., "Solving mixed integer programming problems using automatic reformulation", Operations Research 35 (1987) 45-57.