- Email: [email protected]
constraint logic programming strategies for solving scheduling and combinatorial optimization problems
Computers &Chemical Engineering Computers and Chemical Engineering 24 (2000) 337-343 www.elsevier.com/locate/compchemeng
Hybrid mixed-integer/constraint logic programming strategies for solving scheduling and combinatorial optimization problems Iiro Harjunkoski, Department
of Chemical
Engineering,
Vipul Jain, Ignacio E. Grossmann *
Carnegie
Mellon
University,
5000 Forbes Avenue, Pittsburgh,
PA 15213, USA
Abstract In this paper, we present two methods to overcome the combinatorial complexity when solving large discrete optimization problems. The basic idea relies on combining mixed integer programming (MIP) and constraint logic programming (CLP) to exploit their complementary strengths. These strategies are illustrated in the area of job-shop scheduling and trim-loss problems. Comparisons of the strategies are presented with direct solutions on MIP and CLP problems. 0 2000 Elsevier Science Ltd. All rights reserved. Keywords:
Mixed integer programming; Constraint logic programming; Hybrid MIP/CLP
1. Introduction Scheduling and related combinatorial optimization problems such as the trim loss problem pose significant challenges to mixed-integer linear programming (MILP) modeling approaches, particularly for large sized problems. To reduce the effect of the potential combinatorial explosion, logic based optimization methods have started to emerge. These include generalized disjunctive programming (GDP) (Raman & Grossmann, 1994) and constraint logic programming (CLP) (Hentenryck, 1989). GDP is based on the idea of representing discrete/continuous optimization problems through equations, disjunctions and logic propositions, and in terms of Boolean and continuous variables. CLP is somewhat similar in that it also involves equations, logic statements and disjunctions. Important differences are that CLP has more constructs and supports a compact language that is largely procedural and implicit in nature, while GDP gives rise to declarative models. Also, while GDP can be solved either through branch and bound, or as reformulated MILPs, CLPs are solved through implicit enumeration coupled with domain reduction of variables, which in turn makes use of constraint propagation techniques.
* Corresponding author. Tel.: + l-412-2682230. E-mail address:
[email protected] (1.E Grossmann)
While software has been available for a long time for MILP (OSL, CPLEX, XPRESS-MP), there have been very few tools for logic based propagation methods. It is only in recent years that some of this software is emerging, and perhaps the most popular and successful have been ILOG, ECLiPSe and CHIP for solving CLPs. While for general purpose problems ILOG is only sometimes more effective than MILP, the special version of ILOG for scheduling has proved to be quite efficient, particularly for discrete manufacturing related scheduling problems where feasibility is a major issue. In this paper we examine strategies that can fruitfully combine MILP and CLP. Recently Jain and Grossmann (1999) proposed a hybrid MILP/CLP model for MILP problems that have only a subset of variables in the objective function. These authors proposed a decomposition strategy in which a relaxed MILP is solved at the master level, and a feasibility subproblem is solved with CLP. This approach reduced dramatically the computation time in a scheduling problem with dissimilar parallel machines. The extension and efficiency of this method to other classes of problems has not been investigated. We first present results of two major classes of scheduling and combinatorial problems when solved as MILP and CLP problems. These include the scheduling problem in Jain and Grossmann (1999) with expanded data and the trim-loss problem (Harjunkoski, Westerlund, Porn, & Skrifvars, 1998). The first involves finding a least cost schedule to pro-
0098-1354/00/$ - see front matter 0 2000 Elsevier Science Ltd. All rights reserved. PII: SOO98- 1354(00)00470-l
338
I. Harjunkoski et al. /Computers
and Chemical Engineering 24 (2000) 337-343
cess a set of orders using a set of dissimilar parallel machines. The latter problem, typically found in the paper-converting industry, involves non-convex terms, as well as many integer variables, and can be reformulated as an MILP. As shown in the results, both problems have great difficulty with MIP as the size of the problem increases. With CLP improvements are observed, but exponential behavior is also observed in the larger instances. We report in this paper two hybrid approaches for combining MIP and CLP. In the first we consider a decomposition strategy in which we iterate between an MILP master problem, and a feasibility CLP problem. In the second we reformulate the solution of a bilinear MINLP as the solution of an MILP followed by the iterative solution of a feasibility CLP subproblem. Compared to MILP and CLP the numerical results are very encouraging.
2. Background During the last decades, considerable research has been done on methods for solving MILP problems (Nemhauser & Wolsey, 1988). Perhaps the most significant development in the most past has been new software that can increasingly solve larger problems (e.g. CPLEX, XPRESS-MP). Most of these apply the Branch and Bound method technique for handling the integer variables. The greatest bottleneck with this method arises from the potentially large integer search space that is due to the exponential number of possible combinations. The effectiveness of MILP methods also depends on the size of the linear programming (LP) subproblems, and on the gap between the objective value for the optimal solution and the initial LP subproblem. Approaches to overcome this bottleneck include Branch and Cut algorithms in which extra cuts are added to the MILP to strengthen its relaxation gap (Balas, Ceria & Cornuejols, 1996). CLP uses another technique, constraint propagation (CP) at each node for reducing the domain for all the variables (Marriott & Stuckey, 1999). The variable domains can be continuous, discrete or boolean. If a domain becomes empty it means that the node is infeasible and can be fathomed. If a domain contains more than one element (variable value) branching is performed. CLP was originally developed for solving feasibility problems but has also been extended to solve optimization problems. This is done by expressing the objective function as a constraint that equals an objective variable. When the domain of this objective variable has been explored and all nodes fathomed, the search terminates. CLP methods are often very effective in solving feasibility problems, but the performance in optimization depends highly on the propagation al-
gorithms used and on how well-constrained the problem is. While in MILP all the constraints are evaluated simultaneously, CLP evaluates the effect of constraints sequentially by communicating through domains of the variables. Therefore, loosely constrained problems can be very difficult to solve. On the other hand MILP methods require all constraints to be linear equalities or inequalities, which is not the case for CLP. We will show in the following section how constraint propagation can be effectively used in some classes of MILP problems and some MINLP problems involving bilinearities.
3. Strategies for integration The first method is the one proposed by Jain and Grossmann (1999). It is assumed that the MILP problem can be divided into two subproblems. One subproblem is suitable for the MILP framework, i.e. its variables are present in the objective function and its constraints contribute to tight LP-relaxations. In the other, the subproblem leads to a feasibility problem that may contain constraints that do not affect the LP-relaxations significantly. The former variables and constraints form the core of the MILP model and the latter ones are left to the CLP part. The main difference to earlier approaches is that neither of the subproblems are given in complete form as MILP or CLP problems. The hybrid model, which includes MILP constraints, CLP constraints and equivalence relations is as follows. For the MILP, we have both continuous (U and u) and integer (x and v) sets of variables, where we assume that only u and x have non-zero objective function coefficients. Because of the fact that the CLP and MILP model may have different constraint structures and variable definitions, we define an equivalent set of variables for the CLP part (z&T, t7,jj). The hybrid model is then as follows: min c,‘u + c,‘x
(1)
s.t. Au+Bx+Du+EyIa
(2)
(u, x)+%
(3)
2)
G(z7,2, fi, jj) I 0 UER”,
VER~,XEZ~,
zi, _f, z?, J%D
(4) y~Zy
(5) (6)
The objective function (1) is the same as for the MILP model. The relaxed MILP problem is used to obtain a solution that satisfies the constraint sets (2) and (5) and optimizes the objective function. A partial CLP solution can be derived using the equivalence relations (3) and the feasibility problem for fixed (u, x)
I. Harjunkoski et al. /Computers and Chemical Engineering 24 (2000) 337-343
will verify if a solution satisfying the constraints (4) and (6) can be found. If this is the case, then the optimal solution is found and the algorithm terminates. If the CLP solution is infeasible, then cuts that eliminate as many infeasible solutions as possible, are added to the next relaxed MILP problem. The alternating problems are solved until a feasible CLP solution is obtained as shown in Fig. 1. It should be pointed out that if there is no solution to the relaxed MILP problem, then the original problem is also infeasible. A more detailed description of the method can be found in Jain and Grossmann (1999). We consider another integration approach where the original problem contains only discrete variables and the constraints are linear and bilinear. We first relax the MINLP problem by replacing the bilinearities with new variables which yields an MILP. We then solve a feasibility problem using CLP, where the bilinear terms
of the original problem are matched with the variable values of the MILP. The procedure consists of solving successive CLPs, where infeasible solutions are cut off by increasing the value of the objective to the next level. The bilinear problem is assumed to be as follows: min cTx
(7)
s.t. C b,yk I d
(8)
k
Ax,
v>
XEZX,
Feasible I
I
Optimal solution. STOP. Fig. 1. MILPjCLP decomposition method.
Feasible 4
Increase the objective function value
Solve the correspondmg CLP feasiblity problem with 6x4 objective value (81, (9 : 111,(10)
Iufeasible
Feasible Optimal solution. STOP. Fig. 2. Alternative decomposition method.
I
I
a
(9)
yezy
(10)
where (11)
g ‘(h
Y )lin =
UEZU
Analyze solution and add cuts to the relaxed MILP problem
339
c
ai xi +
7
F
cikuik
(12) (13)
In this problem the objective function (7) contains only one of the variables in the constraints. The objective is followed by linear constraints of the other variable (8). The original constraints (9) consist of linear and bilinear terms of both variables, as seen in (1 l), which makes the separation of constraints difficult. However, the problem can be subdivided by forming the relaxed MILP with the new variable &k that replaces the bilinear term, resulting in constraint (12). This eliminates the need of some transformation techniques, that often expand the problem by bringing in new variables and constraints (Harjunkoski et al., 1998). As seen in Fig. 2, the relaxed MILP can be formed by the objective function (7) followed by the linear relaxation (12) of (9) and the constraints (10) and (13). This generates a solution where the x and u variables have integer values but the fractional integer variable y may not have a feasible value. This provides a relatively tight relaxation and the CLP problem does then a feasibility check for a fixed objective value by considering the constraints (8) (9) (10) and (11). It is very difficult to generate any general cuts for integer variables, and in this case the possible cut would act as a lower bound for the objective function. However, since this lower bound will be active in the following relaxed MILP solution, only a feasibility check is required, which is far more efficient when performed with CLP. Thus, the proposed algorithm starts with a relaxed, relatively tight MILP model and continues with successive CLP solutions. The implementation of the two algorithms was performed with OPL-Studio, CPLEX (ILOG CPLEX, 1999) and ILOG-Solver (ILOG Solver, 1999).
340
I. Harjunkoski et al. / Computers and Chemical Engineering 24 (2000) 337-343
i.start 2 ri ViEZ i.start I di -pz,
Fig. 3. The job-shop scheduling problem. Table 1 Hybrid MIPjCLP solution data Problem 1 1 2 2 3 3 4 4 5 5
Iter
Cuts
MILPa
1 2 1 2 1 2 1 2 1 2
2 1 13 1 23 1 5 13 38 21
1 0 15 0 39 0 7 12 77 39
0.00
0.01 0.45 0.02 5.26 0.02 0.56 0.7 36.04 4.26
CLPb
I
1
/
\ /
4. A scheduling problem We consider the job-shop scheduling problem reported in Jain and Grossmann (1999), which is illustrated in Fig. 3. The hybrid MILP/CLP model is as follows: C C iol mob4
s.t.
tsi 2
tsi
(14)
cim4m
ViGZ
ri
C
pi,,,xi,,,
rnEM
mTMxim= 1
(15) ViEZ
(16)
ViEZ
zxi,,,pi,,,
(22)
i requires t,., ;ieZ
(23)
tsi 2 0
(24) l} VkZ, VmeM
-mp{ri}
VmM
if (xi, = 1) then (zi = m) k&Z, VmgM
z,czM
(25)
ViEZ
(26)
ViEZ
(27)
Viol
(28)
The relaxed MILP problem consists of the objective function (14), the constraints (15) to (18), and (24)(25). The CLP feasibility problem is formed by constraints (19) to (23) and (27) to (28). The robustness of the algorithm is tested in this paper by changing the problem data in Jain and Grossmann (1999) so as to yield process times, which would produce very large models for discretization schemes. All release and due dates are multiplied by 10 and in order to avoid the symmetry, we multiply the durations by 10 and add 3 time units to the first set of data and 7 to the second set of data. Also, an extra 9 was added to the due dates of the first set of data of problem 1 and 2. As can be seen in Table 1, all problems can be solved to optimality with very modest CPU times. These times are very similar to the results reported by Jain and Grossmann. If a discretization method had been used to solve the problems in this paper, it would have led to an exponential increase in time.
5. The trim-loss problem
Fig. 4. The trim-loss, problem.
min
i.duration = pz. ViEZ
i.durationg.2
0.06 0.00 0.1 0.01 0.08 0.22 0.59 0.53
Raw-paper
I
(21)
i.startEZ
0.00 0.00
a CPU-s on a SUN Ultra 60 workstation (CPLEX 6.5). bCPU-s on a SUN Ultra 60 workstation (ILOG Solver and Scheduler 4.4).
I
VieZ
x,6(0,
Set
(20)
The trim-loss problem is shown in Fig. 4 and typically found in the paper-converting industry. The problem involves both theoretical difficulties due to its non-convex nature and a large combinatorial domain because most of the decision variables are required to be integers (Harjunkoski et al., 1998). The problem consists of matching patterns so that the customer demand is satisfied through minimum raw paper usage (or waste production) and number of different patterns. The problem gives rise to the bilinear MINLP: min C (cj*mj + Cj.yj)
(29)
mi,nii,YjjsJ
(17)
B max- AmaxI 1 bivzu I B,,,
(18)
Lnil
(1%
yj I mj I Mj*yi Vje J
icI
5
N,,
VE
J
Vje J
(30) (31) (32)
I. Harjunkoski et al. /Computers
n&order
5
1
mj*nij
2
Vid
ni,max
and Chemical Engineering 24 (2000) 337-343
(33)
jeJ mj,
niie Z+
YjEIo,
(34)
l>
The variable nii defines the pattern appearance, i.e. the number of times product i appears in cutting pattern j, and mj corresponds to the number of times a pattern is used. The binary variable yj represents the solution of a given pattern and is related to corresponding mj through Eq. (32). MIP-methods that make use of exact linearizations of the bilinear constraint (33) often fail to even give an initial solution for large problems within a reasonable time. Thus, this highly constrained problem should benefit from the incorporation of CLP methods. The formulation fulfils the coefficient requirements for the alternative decomposition method and, thus, the problem can be subdivided into a relaxed MILP and CLP feasibility problem as shown in Fig. 2. The bilinearity is replaced with the new discrete variable pij, with which the relaxed MILP model can be written as follows:
341
The following four examples have been solved. The problems are defined by customer orders into which an upper production limit is added just for the purposes of domain reduction. The orders and production specifications are shown below: Example 1 Width 330 360 370 415 435
Torder
12 6 15 6 9
nmax
14 7 17 7 10
Problem parameters: Z = J = 5, N=, = 5, B,,,,, = Ni = 2000, Am,X= 200, Mj = {9,6,6,3,2) 9 {3,5, 5,4, 3}, Cj= 10, ci= 1, Mmin= 10. Example 2
min 1 (cj*mj + Cj*yj) mj.Pijvi
U4nax
(35)
jsJ
Width A_Jmj
-
I c bi*pii5 B_;mj
VjE J
(36)
ieI
2
pij
5
J
(37)
VjE J
(38)
Pi, j s nmax
w-4
Nmax.mj
yj I mj I Mj-y, Qrder
2
1
jsJ
W
This problem provides a fairly tight relaxation, since both mj and pii are integer variables and the only weakness lies in that the no, given by pu/mj, cannot he expected to take integer values, even if the variables pi and mj, take integer values. However, the main difficulty is in the combination of CLP and MILP. It is non-trivial to make any cuts to the pii variables because they are already bound by the demand constraints. One possibility would be to generate cuts for individual pij variables, which may easily result in a suboptimal solution. Also, the effectiveness of such cuts may be very low due to the large number of multiple solutions. Therefore, the CLP problem finds a feasible set of variables that satisfies the constraints (30) to (34). If no feasible solution is found, the objective is increased by a minimum step, in this case by increasing either the sum of yj values or the sum of mj values by one depending on the current values and the coefficients in the objective function.
330 360 380 430 490 530
Torder
8 16 12 7 14 16
nmax
10 18 14 9 16 18
Problem parameters: I = J = 5, N_, = 5, B,,,,, = 2100, A,,, = 200, Mj= (24,21, 15, 11,3}, Ni= (3, 3,4,3,2}, cj = 10, Ci = 1, Mmin= 24. Example 3 Width 290 315 350 455 615
Torder
15 28 21 30 24
nmax
18 32 24 34 27
Problem parameters: Z = J = 6, N,,, = 5, B,,,, = 2200, Amax= 100, Mj= (14, 12,8,7,4,2}, Ni= (2, 3, 3, 5 ,3 ,4), cj= 10, cj= 1, M,,,i, = 15.
I. Harjunkoski et al. /Computers
342
and Chemical Engineering 24 (2000) 337-343
Example 4 Width
%rder
4,max
280 325 395 405 455 515
14 26 27 35 30 32
16 30 31 39 34 37
Problem parameters: Z = J = 7, N,,, = 5, B,,, = 2200, Amax= 200, Mj = {30,21,26,25, 14,9,3}, Ni = (2, 3,3,4, 5,4,4}, cj= 10, ci= 1, Mmin= 38. The problem parameters are following. Z is the number of products in the order and J is the number of different possible cutting patterns. The number of products in a pattern is restricted by N,,, and the maximal width of the pattern equals the width of the raw paper, B IIXIX’Because of some machinery constraints, the width of the waste cannot exceed A,,,, which together with the raw paper width defines the limits for the width of each pattern. Mj and Ni define the variable domains for corresponding mj and nij variables after a domain reduction algorithm and the Mmin expresses the minimum number of patterns that will be needed to satisfy the problem. This can be derived from the previous data. Table 2 Combined
MIP/CLP
solution
data
Example
IterS
Nodesb
CPU-S
Obj
1 1 2 2 3 3 4 4
1400 870 625 13 219 20618295 14 406 044 2 728 933 5 444 136
25 870 625 252 20618 330 151 527 2 128 934 46 222
2.91 265.31 15.46 1465.31 10 842.63 8 508.77 9 884.15 > 50 000
102 102 152 152 245 245 384 384
MILP”
CLPb
Obj
0.04 0.32 0.41 0.01
0.05 0.32 65.04 52.33
(MILP) (CLP) (MILP) (CLP) (MILP) (CLP) (MILP) (CLP) a CLP, number b CLP, number
Table 3 Combined Example 1 2 3 4
of fails. of choice point.
MIPjCLP
solution
CLP-Iter 1 1 5 3
data
a CPU-s on a SUN Ultra 60 workstation (CPLEX 6.5). b CPU-s on a SUN Ultra 60 workstation (ILOG Solver Scheduler 4.4).
102 152 245 384
and
The cj and Cj are the cost coefficients for each pattern j. Table 2 shows the results when MILP and CLP are used to solve the bilinear problem. In the use of MILP exact linearizations are performed. In the case of CLP the bilinearity in (29)-(34) are handled directly by ILOG. The results of the alternative decomposition method are shown in Table 3. As can be observed in Table 2, both CLP and MILP are able to solve the problems for the problems but only after considerably long solution times. CLP found the optimal solution for example 4 but could not verify the optimality. On the other hand, as seen in Table 3, the hybrid strategy solves the problem very efficiently. It should be noted that the CLP-time given in the table refers only to the last feasible iteration. The CLP-part of the problem might easily suffer from weak constraints and the performance is much worse if for instance the upper production limits or other tightening constraints are removed. On the other hand, removing the active lower bound Mmi,, only slightly increases the total solution time for the hybrid model because of a tight initial MILP problem.
6. Conclusions The main objective of this paper was to consider two different methods of using a hybrid CLP/MILP model for solving problems that are very hard to solve with either of the methods alone. First the approach by Jain and Grossmann (1999) was revisited and some modifications were made to the original problem data. The algorithm proved to be robust, even if the timescale was increased by an order of magnitude. In a uniform time discretization approach this would have complicated the problem significantly. However, the hybrid model could be solved as efficiently as with the original smaller data. The second example presented another way of combining the CLP and MILP methods. First, a relaxed MILP problem was solved to provide a tight lower bound for the original problem. This was followed by a sequential feasibility problem solved by CLP. The CLP problem consisted basically of the original bilinear MINLP problem with a fixed objective value. The results show that CLP can be used to partly overcome the critical problem caused by the large combinatorial space. It should be noted that regardless of the strategy, all solutions methods can obtain the global optimal solution to the original non-convex INLP problem. The combination of CLP and MILP methods may result in significant improvements in the performance of solving large problems containing mainly or only in-
I. Harjunkoski
et al. /Computers
and Chemical Engineering 24 (2000) 337-343
teger variable. Finding a general and effective method to connect the MILP and CLP can be very difficult, since each step in one method should reduce the search domain also for the other. The methods presented here can, of course, be applied to other similar type of problems and future research will focus on expanding them to more general cases. However, since CLP is highly dependent on the constraining level, the numerical properties of the actual problem and its relaxation are of great importance for the performance of these approaches.
Acknowledgements Svenska Tekniska Forskningsakademien i Finland (Swedish Academy of Engineering Sciences in Finland) and the Center for Advanced Process Decision-making at Carnegie Mellon University are gratefully acknowledged for their financial support.
343
References Balas, E., Ceria, S., & Comuejols, G. (1996). Mixed O-l programming by lift-and-project in a branch-and-cut framework. Management Science, 42, 1229- 1246.
Harjunkoski, I., Westerlund, T., Porn, R., & Skrifvars, H. (1998). Different transformations for solving nonconvex trim-loss problems by MINLP. European Journal of Operational Research, 105, 594-603.
Hentenryck, P. V. (1989). Constraint satisfaction in logic programming. Cambridge, MA: MIT Press. ILOG CPLEX 6.5. (1999). User’s Manual. ILOG Inc. ILOG Solver 4.4. User’s Manual (1999). ILOG Inc. Jain, V., & Grossmann, I.E. (1999). Algorithms for hybrid MILP/ CLP models for a class of optimization problems (submitted for publication). Marriott, K., & Stuckey, P. J. (1999). Introduction to constrainf logic programming. Cambridge: MIT press. Nemhauser, G. L., & Wolsey, L. A. (1988). Integer and combinatorial optimization. New York: Wiley. Raman, R., & Grossmann, I. E. (1994). Modelling and computational techniques for logic based integer programming. Computers & Chemical Engineering,
18, 563-578.
Hybrid mixed-integer/constraint logic programming strategies for solving scheduling and combinatorial optimization problems Iiro Harjunkoski, Department
of Chemical
Engineering,
Vipul Jain, Ignacio E. Grossmann *
Carnegie
Mellon
University,
5000 Forbes Avenue, Pittsburgh,
PA 15213, USA
Abstract In this paper, we present two methods to overcome the combinatorial complexity when solving large discrete optimization problems. The basic idea relies on combining mixed integer programming (MIP) and constraint logic programming (CLP) to exploit their complementary strengths. These strategies are illustrated in the area of job-shop scheduling and trim-loss problems. Comparisons of the strategies are presented with direct solutions on MIP and CLP problems. 0 2000 Elsevier Science Ltd. All rights reserved. Keywords:
Mixed integer programming; Constraint logic programming; Hybrid MIP/CLP
1. Introduction Scheduling and related combinatorial optimization problems such as the trim loss problem pose significant challenges to mixed-integer linear programming (MILP) modeling approaches, particularly for large sized problems. To reduce the effect of the potential combinatorial explosion, logic based optimization methods have started to emerge. These include generalized disjunctive programming (GDP) (Raman & Grossmann, 1994) and constraint logic programming (CLP) (Hentenryck, 1989). GDP is based on the idea of representing discrete/continuous optimization problems through equations, disjunctions and logic propositions, and in terms of Boolean and continuous variables. CLP is somewhat similar in that it also involves equations, logic statements and disjunctions. Important differences are that CLP has more constructs and supports a compact language that is largely procedural and implicit in nature, while GDP gives rise to declarative models. Also, while GDP can be solved either through branch and bound, or as reformulated MILPs, CLPs are solved through implicit enumeration coupled with domain reduction of variables, which in turn makes use of constraint propagation techniques.
* Corresponding author. Tel.: + l-412-2682230. E-mail address:
[email protected] (1.E Grossmann)
While software has been available for a long time for MILP (OSL, CPLEX, XPRESS-MP), there have been very few tools for logic based propagation methods. It is only in recent years that some of this software is emerging, and perhaps the most popular and successful have been ILOG, ECLiPSe and CHIP for solving CLPs. While for general purpose problems ILOG is only sometimes more effective than MILP, the special version of ILOG for scheduling has proved to be quite efficient, particularly for discrete manufacturing related scheduling problems where feasibility is a major issue. In this paper we examine strategies that can fruitfully combine MILP and CLP. Recently Jain and Grossmann (1999) proposed a hybrid MILP/CLP model for MILP problems that have only a subset of variables in the objective function. These authors proposed a decomposition strategy in which a relaxed MILP is solved at the master level, and a feasibility subproblem is solved with CLP. This approach reduced dramatically the computation time in a scheduling problem with dissimilar parallel machines. The extension and efficiency of this method to other classes of problems has not been investigated. We first present results of two major classes of scheduling and combinatorial problems when solved as MILP and CLP problems. These include the scheduling problem in Jain and Grossmann (1999) with expanded data and the trim-loss problem (Harjunkoski, Westerlund, Porn, & Skrifvars, 1998). The first involves finding a least cost schedule to pro-
0098-1354/00/$ - see front matter 0 2000 Elsevier Science Ltd. All rights reserved. PII: SOO98- 1354(00)00470-l
338
I. Harjunkoski et al. /Computers
and Chemical Engineering 24 (2000) 337-343
cess a set of orders using a set of dissimilar parallel machines. The latter problem, typically found in the paper-converting industry, involves non-convex terms, as well as many integer variables, and can be reformulated as an MILP. As shown in the results, both problems have great difficulty with MIP as the size of the problem increases. With CLP improvements are observed, but exponential behavior is also observed in the larger instances. We report in this paper two hybrid approaches for combining MIP and CLP. In the first we consider a decomposition strategy in which we iterate between an MILP master problem, and a feasibility CLP problem. In the second we reformulate the solution of a bilinear MINLP as the solution of an MILP followed by the iterative solution of a feasibility CLP subproblem. Compared to MILP and CLP the numerical results are very encouraging.
2. Background During the last decades, considerable research has been done on methods for solving MILP problems (Nemhauser & Wolsey, 1988). Perhaps the most significant development in the most past has been new software that can increasingly solve larger problems (e.g. CPLEX, XPRESS-MP). Most of these apply the Branch and Bound method technique for handling the integer variables. The greatest bottleneck with this method arises from the potentially large integer search space that is due to the exponential number of possible combinations. The effectiveness of MILP methods also depends on the size of the linear programming (LP) subproblems, and on the gap between the objective value for the optimal solution and the initial LP subproblem. Approaches to overcome this bottleneck include Branch and Cut algorithms in which extra cuts are added to the MILP to strengthen its relaxation gap (Balas, Ceria & Cornuejols, 1996). CLP uses another technique, constraint propagation (CP) at each node for reducing the domain for all the variables (Marriott & Stuckey, 1999). The variable domains can be continuous, discrete or boolean. If a domain becomes empty it means that the node is infeasible and can be fathomed. If a domain contains more than one element (variable value) branching is performed. CLP was originally developed for solving feasibility problems but has also been extended to solve optimization problems. This is done by expressing the objective function as a constraint that equals an objective variable. When the domain of this objective variable has been explored and all nodes fathomed, the search terminates. CLP methods are often very effective in solving feasibility problems, but the performance in optimization depends highly on the propagation al-
gorithms used and on how well-constrained the problem is. While in MILP all the constraints are evaluated simultaneously, CLP evaluates the effect of constraints sequentially by communicating through domains of the variables. Therefore, loosely constrained problems can be very difficult to solve. On the other hand MILP methods require all constraints to be linear equalities or inequalities, which is not the case for CLP. We will show in the following section how constraint propagation can be effectively used in some classes of MILP problems and some MINLP problems involving bilinearities.
3. Strategies for integration The first method is the one proposed by Jain and Grossmann (1999). It is assumed that the MILP problem can be divided into two subproblems. One subproblem is suitable for the MILP framework, i.e. its variables are present in the objective function and its constraints contribute to tight LP-relaxations. In the other, the subproblem leads to a feasibility problem that may contain constraints that do not affect the LP-relaxations significantly. The former variables and constraints form the core of the MILP model and the latter ones are left to the CLP part. The main difference to earlier approaches is that neither of the subproblems are given in complete form as MILP or CLP problems. The hybrid model, which includes MILP constraints, CLP constraints and equivalence relations is as follows. For the MILP, we have both continuous (U and u) and integer (x and v) sets of variables, where we assume that only u and x have non-zero objective function coefficients. Because of the fact that the CLP and MILP model may have different constraint structures and variable definitions, we define an equivalent set of variables for the CLP part (z&T, t7,jj). The hybrid model is then as follows: min c,‘u + c,‘x
(1)
s.t. Au+Bx+Du+EyIa
(2)
(u, x)+%
(3)
2)
G(z7,2, fi, jj) I 0 UER”,
VER~,XEZ~,
zi, _f, z?, J%D
(4) y~Zy
(5) (6)
The objective function (1) is the same as for the MILP model. The relaxed MILP problem is used to obtain a solution that satisfies the constraint sets (2) and (5) and optimizes the objective function. A partial CLP solution can be derived using the equivalence relations (3) and the feasibility problem for fixed (u, x)
I. Harjunkoski et al. /Computers and Chemical Engineering 24 (2000) 337-343
will verify if a solution satisfying the constraints (4) and (6) can be found. If this is the case, then the optimal solution is found and the algorithm terminates. If the CLP solution is infeasible, then cuts that eliminate as many infeasible solutions as possible, are added to the next relaxed MILP problem. The alternating problems are solved until a feasible CLP solution is obtained as shown in Fig. 1. It should be pointed out that if there is no solution to the relaxed MILP problem, then the original problem is also infeasible. A more detailed description of the method can be found in Jain and Grossmann (1999). We consider another integration approach where the original problem contains only discrete variables and the constraints are linear and bilinear. We first relax the MINLP problem by replacing the bilinearities with new variables which yields an MILP. We then solve a feasibility problem using CLP, where the bilinear terms
of the original problem are matched with the variable values of the MILP. The procedure consists of solving successive CLPs, where infeasible solutions are cut off by increasing the value of the objective to the next level. The bilinear problem is assumed to be as follows: min cTx
(7)
s.t. C b,yk I d
(8)
k
Ax,
v>
XEZX,
Feasible I
I
Optimal solution. STOP. Fig. 1. MILPjCLP decomposition method.
Feasible 4
Increase the objective function value
Solve the correspondmg CLP feasiblity problem with 6x4 objective value (81, (9 : 111,(10)
Iufeasible
Feasible Optimal solution. STOP. Fig. 2. Alternative decomposition method.
I
I
a
(9)
yezy
(10)
where (11)
g ‘(h
Y )lin =
UEZU
Analyze solution and add cuts to the relaxed MILP problem
339
c
ai xi +
7
F
cikuik
(12) (13)
In this problem the objective function (7) contains only one of the variables in the constraints. The objective is followed by linear constraints of the other variable (8). The original constraints (9) consist of linear and bilinear terms of both variables, as seen in (1 l), which makes the separation of constraints difficult. However, the problem can be subdivided by forming the relaxed MILP with the new variable &k that replaces the bilinear term, resulting in constraint (12). This eliminates the need of some transformation techniques, that often expand the problem by bringing in new variables and constraints (Harjunkoski et al., 1998). As seen in Fig. 2, the relaxed MILP can be formed by the objective function (7) followed by the linear relaxation (12) of (9) and the constraints (10) and (13). This generates a solution where the x and u variables have integer values but the fractional integer variable y may not have a feasible value. This provides a relatively tight relaxation and the CLP problem does then a feasibility check for a fixed objective value by considering the constraints (8) (9) (10) and (11). It is very difficult to generate any general cuts for integer variables, and in this case the possible cut would act as a lower bound for the objective function. However, since this lower bound will be active in the following relaxed MILP solution, only a feasibility check is required, which is far more efficient when performed with CLP. Thus, the proposed algorithm starts with a relaxed, relatively tight MILP model and continues with successive CLP solutions. The implementation of the two algorithms was performed with OPL-Studio, CPLEX (ILOG CPLEX, 1999) and ILOG-Solver (ILOG Solver, 1999).
340
I. Harjunkoski et al. / Computers and Chemical Engineering 24 (2000) 337-343
i.start 2 ri ViEZ i.start I di -pz,
Fig. 3. The job-shop scheduling problem. Table 1 Hybrid MIPjCLP solution data Problem 1 1 2 2 3 3 4 4 5 5
Iter
Cuts
MILPa
1 2 1 2 1 2 1 2 1 2
2 1 13 1 23 1 5 13 38 21
1 0 15 0 39 0 7 12 77 39
0.00
0.01 0.45 0.02 5.26 0.02 0.56 0.7 36.04 4.26
CLPb
I
1
/
\ /
4. A scheduling problem We consider the job-shop scheduling problem reported in Jain and Grossmann (1999), which is illustrated in Fig. 3. The hybrid MILP/CLP model is as follows: C C iol mob4
s.t.
tsi 2
tsi
(14)
cim4m
ViGZ
ri
C
pi,,,xi,,,
rnEM
mTMxim= 1
(15) ViEZ
(16)
ViEZ
zxi,,,pi,,,
(22)
i requires t,., ;ieZ
(23)
tsi 2 0
(24) l} VkZ, VmeM
-mp{ri}
VmM
if (xi, = 1) then (zi = m) k&Z, VmgM
z,czM
(25)
ViEZ
(26)
ViEZ
(27)
Viol
(28)
The relaxed MILP problem consists of the objective function (14), the constraints (15) to (18), and (24)(25). The CLP feasibility problem is formed by constraints (19) to (23) and (27) to (28). The robustness of the algorithm is tested in this paper by changing the problem data in Jain and Grossmann (1999) so as to yield process times, which would produce very large models for discretization schemes. All release and due dates are multiplied by 10 and in order to avoid the symmetry, we multiply the durations by 10 and add 3 time units to the first set of data and 7 to the second set of data. Also, an extra 9 was added to the due dates of the first set of data of problem 1 and 2. As can be seen in Table 1, all problems can be solved to optimality with very modest CPU times. These times are very similar to the results reported by Jain and Grossmann. If a discretization method had been used to solve the problems in this paper, it would have led to an exponential increase in time.
5. The trim-loss problem
Fig. 4. The trim-loss, problem.
min
i.duration = pz. ViEZ
i.durationg.2
0.06 0.00 0.1 0.01 0.08 0.22 0.59 0.53
Raw-paper
I
(21)
i.startEZ
0.00 0.00
a CPU-s on a SUN Ultra 60 workstation (CPLEX 6.5). bCPU-s on a SUN Ultra 60 workstation (ILOG Solver and Scheduler 4.4).
I
VieZ
x,6(0,
Set
(20)
The trim-loss problem is shown in Fig. 4 and typically found in the paper-converting industry. The problem involves both theoretical difficulties due to its non-convex nature and a large combinatorial domain because most of the decision variables are required to be integers (Harjunkoski et al., 1998). The problem consists of matching patterns so that the customer demand is satisfied through minimum raw paper usage (or waste production) and number of different patterns. The problem gives rise to the bilinear MINLP: min C (cj*mj + Cj.yj)
(29)
mi,nii,YjjsJ
(17)
B max- AmaxI 1 bivzu I B,,,
(18)
Lnil
(1%
yj I mj I Mj*yi Vje J
icI
5
N,,
VE
J
Vje J
(30) (31) (32)
I. Harjunkoski et al. /Computers
n&order
5
1
mj*nij
2
Vid
ni,max
and Chemical Engineering 24 (2000) 337-343
(33)
jeJ mj,
niie Z+
YjEIo,
(34)
l>
The variable nii defines the pattern appearance, i.e. the number of times product i appears in cutting pattern j, and mj corresponds to the number of times a pattern is used. The binary variable yj represents the solution of a given pattern and is related to corresponding mj through Eq. (32). MIP-methods that make use of exact linearizations of the bilinear constraint (33) often fail to even give an initial solution for large problems within a reasonable time. Thus, this highly constrained problem should benefit from the incorporation of CLP methods. The formulation fulfils the coefficient requirements for the alternative decomposition method and, thus, the problem can be subdivided into a relaxed MILP and CLP feasibility problem as shown in Fig. 2. The bilinearity is replaced with the new discrete variable pij, with which the relaxed MILP model can be written as follows:
341
The following four examples have been solved. The problems are defined by customer orders into which an upper production limit is added just for the purposes of domain reduction. The orders and production specifications are shown below: Example 1 Width 330 360 370 415 435
Torder
12 6 15 6 9
nmax
14 7 17 7 10
Problem parameters: Z = J = 5, N=, = 5, B,,,,, = Ni = 2000, Am,X= 200, Mj = {9,6,6,3,2) 9 {3,5, 5,4, 3}, Cj= 10, ci= 1, Mmin= 10. Example 2
min 1 (cj*mj + Cj*yj) mj.Pijvi
U4nax
(35)
jsJ
Width A_Jmj
-
I c bi*pii5 B_;mj
VjE J
(36)
ieI
2
pij
5
J
(37)
VjE J
(38)
Pi, j s nmax
w-4
Nmax.mj
yj I mj I Mj-y, Qrder
2
1
jsJ
W
This problem provides a fairly tight relaxation, since both mj and pii are integer variables and the only weakness lies in that the no, given by pu/mj, cannot he expected to take integer values, even if the variables pi and mj, take integer values. However, the main difficulty is in the combination of CLP and MILP. It is non-trivial to make any cuts to the pii variables because they are already bound by the demand constraints. One possibility would be to generate cuts for individual pij variables, which may easily result in a suboptimal solution. Also, the effectiveness of such cuts may be very low due to the large number of multiple solutions. Therefore, the CLP problem finds a feasible set of variables that satisfies the constraints (30) to (34). If no feasible solution is found, the objective is increased by a minimum step, in this case by increasing either the sum of yj values or the sum of mj values by one depending on the current values and the coefficients in the objective function.
330 360 380 430 490 530
Torder
8 16 12 7 14 16
nmax
10 18 14 9 16 18
Problem parameters: I = J = 5, N_, = 5, B,,,,, = 2100, A,,, = 200, Mj= (24,21, 15, 11,3}, Ni= (3, 3,4,3,2}, cj = 10, Ci = 1, Mmin= 24. Example 3 Width 290 315 350 455 615
Torder
15 28 21 30 24
nmax
18 32 24 34 27
Problem parameters: Z = J = 6, N,,, = 5, B,,,, = 2200, Amax= 100, Mj= (14, 12,8,7,4,2}, Ni= (2, 3, 3, 5 ,3 ,4), cj= 10, cj= 1, M,,,i, = 15.
I. Harjunkoski et al. /Computers
342
and Chemical Engineering 24 (2000) 337-343
Example 4 Width
%rder
4,max
280 325 395 405 455 515
14 26 27 35 30 32
16 30 31 39 34 37
Problem parameters: Z = J = 7, N,,, = 5, B,,, = 2200, Amax= 200, Mj = {30,21,26,25, 14,9,3}, Ni = (2, 3,3,4, 5,4,4}, cj= 10, ci= 1, Mmin= 38. The problem parameters are following. Z is the number of products in the order and J is the number of different possible cutting patterns. The number of products in a pattern is restricted by N,,, and the maximal width of the pattern equals the width of the raw paper, B IIXIX’Because of some machinery constraints, the width of the waste cannot exceed A,,,, which together with the raw paper width defines the limits for the width of each pattern. Mj and Ni define the variable domains for corresponding mj and nij variables after a domain reduction algorithm and the Mmin expresses the minimum number of patterns that will be needed to satisfy the problem. This can be derived from the previous data. Table 2 Combined
MIP/CLP
solution
data
Example
IterS
Nodesb
CPU-S
Obj
1 1 2 2 3 3 4 4
1400 870 625 13 219 20618295 14 406 044 2 728 933 5 444 136
25 870 625 252 20618 330 151 527 2 128 934 46 222
2.91 265.31 15.46 1465.31 10 842.63 8 508.77 9 884.15 > 50 000
102 102 152 152 245 245 384 384
MILP”
CLPb
Obj
0.04 0.32 0.41 0.01
0.05 0.32 65.04 52.33
(MILP) (CLP) (MILP) (CLP) (MILP) (CLP) (MILP) (CLP) a CLP, number b CLP, number
Table 3 Combined Example 1 2 3 4
of fails. of choice point.
MIPjCLP
solution
CLP-Iter 1 1 5 3
data
a CPU-s on a SUN Ultra 60 workstation (CPLEX 6.5). b CPU-s on a SUN Ultra 60 workstation (ILOG Solver Scheduler 4.4).
102 152 245 384
and
The cj and Cj are the cost coefficients for each pattern j. Table 2 shows the results when MILP and CLP are used to solve the bilinear problem. In the use of MILP exact linearizations are performed. In the case of CLP the bilinearity in (29)-(34) are handled directly by ILOG. The results of the alternative decomposition method are shown in Table 3. As can be observed in Table 2, both CLP and MILP are able to solve the problems for the problems but only after considerably long solution times. CLP found the optimal solution for example 4 but could not verify the optimality. On the other hand, as seen in Table 3, the hybrid strategy solves the problem very efficiently. It should be noted that the CLP-time given in the table refers only to the last feasible iteration. The CLP-part of the problem might easily suffer from weak constraints and the performance is much worse if for instance the upper production limits or other tightening constraints are removed. On the other hand, removing the active lower bound Mmi,, only slightly increases the total solution time for the hybrid model because of a tight initial MILP problem.
6. Conclusions The main objective of this paper was to consider two different methods of using a hybrid CLP/MILP model for solving problems that are very hard to solve with either of the methods alone. First the approach by Jain and Grossmann (1999) was revisited and some modifications were made to the original problem data. The algorithm proved to be robust, even if the timescale was increased by an order of magnitude. In a uniform time discretization approach this would have complicated the problem significantly. However, the hybrid model could be solved as efficiently as with the original smaller data. The second example presented another way of combining the CLP and MILP methods. First, a relaxed MILP problem was solved to provide a tight lower bound for the original problem. This was followed by a sequential feasibility problem solved by CLP. The CLP problem consisted basically of the original bilinear MINLP problem with a fixed objective value. The results show that CLP can be used to partly overcome the critical problem caused by the large combinatorial space. It should be noted that regardless of the strategy, all solutions methods can obtain the global optimal solution to the original non-convex INLP problem. The combination of CLP and MILP methods may result in significant improvements in the performance of solving large problems containing mainly or only in-
I. Harjunkoski
et al. /Computers
and Chemical Engineering 24 (2000) 337-343
teger variable. Finding a general and effective method to connect the MILP and CLP can be very difficult, since each step in one method should reduce the search domain also for the other. The methods presented here can, of course, be applied to other similar type of problems and future research will focus on expanding them to more general cases. However, since CLP is highly dependent on the constraining level, the numerical properties of the actual problem and its relaxation are of great importance for the performance of these approaches.
Acknowledgements Svenska Tekniska Forskningsakademien i Finland (Swedish Academy of Engineering Sciences in Finland) and the Center for Advanced Process Decision-making at Carnegie Mellon University are gratefully acknowledged for their financial support.
343
References Balas, E., Ceria, S., & Comuejols, G. (1996). Mixed O-l programming by lift-and-project in a branch-and-cut framework. Management Science, 42, 1229- 1246.
Harjunkoski, I., Westerlund, T., Porn, R., & Skrifvars, H. (1998). Different transformations for solving nonconvex trim-loss problems by MINLP. European Journal of Operational Research, 105, 594-603.
Hentenryck, P. V. (1989). Constraint satisfaction in logic programming. Cambridge, MA: MIT Press. ILOG CPLEX 6.5. (1999). User’s Manual. ILOG Inc. ILOG Solver 4.4. User’s Manual (1999). ILOG Inc. Jain, V., & Grossmann, I.E. (1999). Algorithms for hybrid MILP/ CLP models for a class of optimization problems (submitted for publication). Marriott, K., & Stuckey, P. J. (1999). Introduction to constrainf logic programming. Cambridge: MIT press. Nemhauser, G. L., & Wolsey, L. A. (1988). Integer and combinatorial optimization. New York: Wiley. Raman, R., & Grossmann, I. E. (1994). Modelling and computational techniques for logic based integer programming. Computers & Chemical Engineering,
18, 563-578.