Accelerating column generation for variable sized bin-packing problems

European Journal of Operational Research 183 (2007) 1333–1352 www.elsevier.com/locate/ejor

Accelerating column generation for variable sized bin-packing problems Cla´udio Alves *, J.M. Vale´rio de Carvalho Departamento de Produc¸a˜o e Sistemas, Escola de Engenharia, Universidade do Minho, 4710-057 Braga, Portugal Received 30 July 2004; accepted 26 July 2005 Available online 12 June 2006

Abstract In this paper, we study different strategies to stabilize and accelerate the column generation method, when it is applied specifically to the variable sized bin-packing problem, or to its cutting stock counterpart, the multiple length cutting stock problem. Many of the algorithms for these problems discussed in the literature rely on column generation, processes that are known to converge slowly due to primal degeneracy and the excessive oscillations of the dual variables. In the sequel, we introduce new dual-optimal inequalities, and explore the principle of model aggregation as an alternative way of controlling the progress of the dual variables. Two algorithms based on aggregation are proposed. The first one relies on a row aggregated LP, while the second one solves iteratively sequences of doubly aggregated models. Working with these approximations, in the various stages of an iterative solution process, has proven to be an effective way of achieving faster convergence. The computational experiments were conducted on a broad range of instances, many of them published in the literature. They show a significant reduction of the number of column generation iterations and computing time.  2006 Elsevier B.V. All rights reserved. Keywords: Integer programming; Column generation; Variable sized bin-packing; Multiple length cutting stock; Convergence

1. Introduction Packing and cutting problems share the same objective: find the best way of assigning a set of objects to other ones. Many variations are allowed. Dyckhoff [11] suggests a typology in which these problems are classified mainly according to their dimensionality, the shape of the objects, the types of assortments and the availability of large and small objects. This latter point distinguishes bin-packing from cutting stock problems. In a bin-packing problem, we have to deal with a limited number of small objects, whereas the cutting stock problems involve quantities that are far greater than the unit. In this paper, we consider the one-dimensional variable sized bin-packing problem, which is characterized by bins with different capacities, and a cost function that depends on the total capacity required. We address in *

Corresponding author. Tel.: +351 253 604751; fax: +351 253 604741. E-mail address: [email protected] (C. Alves).

0377-2217/$ - see front matter  2006 Elsevier B.V. All rights reserved. doi:10.1016/j.ejor.2005.07.033

1334

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

particular the case in which the number of bins is bounded. To clarify the presentation, we will refer only to the variable sized bin-packing problem, but note that the approaches we devised are identically applicable to the multiple length cutting stock problem, its cutting stock counterpart. Indeed, no special attention was given to the quantities that are ordered. The problem is a generalization of the standard bin-packing problem, which deserved in the past much more attention from the research community. It is NP-hard [13]; no absolute approximation scheme can be devised that solves it in fully polynomial time. Therefore, a lot of research has been initially devoted to the development of approximation algorithms for which only worst-case performance could be guaranteed. Friesen and Langston [12], Murgolo [24], Chu and La [6] worked on such algorithms. Meanwhile, solutions to the multiple length cutting stock problem were proposed. Gilmore and Gomory [15] extend their column generation scheme for the cutting stock problem to what they call the machine balance problem. Roodman [26] describes how to find near optimal solutions starting with the linear programming (LP) relaxation, and generating columns heuristically. Recently, Holthaus [17] reports extensive computational experiments on combining column generation with different methods to solve the residual problem arising from the rounding of the continuous solution. Only recently, attempts have been made to construct efficient algorithms that solve the variable sized binpacking problem to optimality. Monaci [23] devises an exact branch-and-bound algorithm based on a branching rule from Martello and Toth [22]. Lower bounds are computed along the tree, one of which through a column generation procedure. Belov and Scheithauer [3] use Chva´tal-Gomory cutting planes embedded in a column generation algorithm to solve the multiple length cutting stock problem. Their pricing subproblem becomes a general integer program without any special structure that has to be solved by branch-and-bound. Alves and Vale´rio de Carvalho [1] propose a branch-and-price algorithm where branching constraints are imposed on the arc-flow variables of an original compact formulation. The subproblem remains a knapsack problem, solvable in pseudo-polynomial time. In fact, column generation is at the heart of most of the solution approaches. The model it solves, that results from the Dantzig–Wolfe decomposition of a compact formulation [32,30], is stronger and leads to improved lower bounds. However, it is well known that column generation procedures suffer from slow convergence induced by undesirable behaviors such as primal degeneracy [8], or the excessive oscillations of the dual variables [10]. In the last years, many efforts have been devoted to the topic of stabilized column generation, with the purpose of accelerating these processes. One way of accelerating column generation was proposed by Degraeve and Peeters [7], who use a simplex method/subgradient-optimization procedure to solve the LP relaxation of the cutting stock problem. To obtain the optimal solution, the following procedure is repeated: for a specific number of iterations, subgradient optimization is used to update the dual prices, and new columns are priced and generated; then, the simplex method is used to reoptimize the master problem with the new columns added. Column generation and Lagrangean relaxation are equivalent [25], and the subgradient method has been extensively used to solve the Lagrangean problem. Column generation is known to be more robust, but it has the burden of reoptimizing the master problem to update the dual variables at every iteration. The subgradient method provides a fast way of updating the dual solution, but may have some convergence difficulties. The hybrid procedure of Degraeve and Peeters combines the robustness of the column generation method with the fast update of the dual prices of the subgradient method, producing an overall acceleration of the solution process. It will become clear that the procedures presented in this paper, which are based on a different idea, can be combined with the one used by Degraeve and Peeters. Stabilization is a different way of accelerating the column generation process. Usually, stabilization may be achieved by restricting once the admissible dual solution space or, instead, by guiding the progress of the dual variables. The boxstep method of Marsten et al. [21] follows this latter strategy by drawing fixed-size boxes around the solutions of the dual restricted master problem. The trust region method [19] uses a similar concept but relies on box constraints whose sizes may be dynamically updated. From the primal standpoint, these methods solve successively a restricted master problem in which slack and surplus variables are penalized. In [10], du Merle et al. extend this approach by imposing additional constraints to these variables. They suggest strategies to set the box sizes, and report promising results on air transportation and location problems. Other methods, such as bundle [16] and analytic center cutting plane methods [9], have been used to prevent the excessive variations of the dual variables.

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1335

Recent publications have shown how to reduce the instability of column generation, when this method is applied to bin-packing or cutting stock problems. Vale´rio de Carvalho [31] proposes adding a polynomial number of columns (i.e., dual inequalities) prior to the solution of the first restricted master problem. He derives a family of dual inequalities, and proves that they are weak dual-optimal inequalities in the sense introduced by Ben Amor et al. [5] that they do not exclude any dual-optimal solution. Since primal feasibility may be lost, Ben Amor et al. [5] suggest perturbing the right hand side of the dual inequalities by small amounts, forcing those columns to have null values in any optimal solution. Besides, they show that the aggregation into single constraints of items with the same size leads to a substantial acceleration of the overall solution process. In their models for packing and cutting problems, researchers have since long considered this aggregation. This was already a way of implicitly restricting the dual space by enforcing equality between some dual variables. In fact, we know that there always exists an optimal solution in which items of identical sizes have corresponding dual variables with the same value. In practice, this equality extends frequently to other items with nearly the same size. This phenomenon was early pointed out by Gilmore and Gomory [15], who explore it only to reduce the size of the knapsack subproblems. In this paper, we present different strategies to stabilize and accelerate column generation for the variable sized bin-packing problem. We concentrate on the LP relaxation. Accelerating the resolution of the LP relaxation is an important matter, since this a way of getting good lower bounds in less time. However, being able to apply these stabilization techniques to all the nodes of a branching tree is even more interesting. Indeed, in a branch-and-price framework the tailing off effect does not occur only at the root node. In a forthcoming paper, we will show how to use one of the stabilization techniques discussed below, namely the dual inequalities, to the whole branch-and-bound tree [1]. In the sequel, new dual-optimal inequalities will be analyzed. We discuss their relative strength, and proceed by exploring the principle of model aggregation as an alternative way of controlling the progress of the dual variables. Two algorithms based on the iterative resolution of aggregated models are proposed. The first is a simple two-steps procedure that starts by solving a row aggregated LP resulting from the juxtaposition of the original items. The second is an iterative algorithm that solves a sequence of smaller size approximations obtained through a double aggregation of variables and constraints. This aggregation scheme amounts to imposing equality constraints between some dual variables. Note that these aggregation strategies rely essentially on items, which allows us to use them for the single sized bin-packing problem. This paper is organized as follows. The mathematical formulation for the variable sized bin-packing problem is presented in Section 2. In Section 3, we briefly recall the dual inequalities introduced in [31], which are still valid even when bins with different capacities are available, and we describe the set of new dual inequalities. Section 4 is devoted to our aggregation based procedures. Extensive computational results are presented in Section 5. 2. Variable sized bin-packing problems: A mathematical formulation In variable sized bin-packing problems, we are given K classes of bins consisting in Bk bins of integer capacity Wk, such that W k 6¼ W k0 , for all k 5 k 0 , and m different sets of items, each one with its own integer size wi, i = 1, . . . , m. The number of items to pack, or demand, in each set i is denoted by bi. Throughout this paper, we will assume that the items and bins are sorted in order of decreasing sizes and capacities, respectively. The objective is to find what packing patterns to use, and in what extent, in order to satisfy at minimum cost the demand using nothing more than the available bins. The cost of a solution is expressed as the total capacity required to pack the items. So, finding a minimum cost solution amounts to minimizing the total unfulfilled space. Note that, due to the availability constraints, the problem may be infeasible. Using the new typology developed by Wa¨scher et al. [34], we can characterize this problem as a multiple bin size bin packing problem, with a single and fixed dimension, a weakly heterogeneous assortment of large objects, a strongly heterogeneous assortment of small items, and an objective that consists on input minimization. The problem is formulated through the following integer linear programming (ILP) model: min

pk K X X k¼1 r¼1

W k xkr ;

ð1Þ

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1336

subject to

pk K X X

i ¼ 1; . . . ; m;

aikr xkr P bi ;

k¼1 r¼1 pk X

xkr 6 Bk ;

k ¼ 1; . . . ; K;

ð2Þ ð3Þ

r¼1

xkr P 0; and integer;

k ¼ 1; . . . ; K; r ¼ 1; . . . ; pk :

ð4Þ

Formulation (1)–(4) has an exponential number of columns, denoted by p. Each column r, r = 1, . . . , p, consists in a vector (a1kr, a2kr, . . . , amkr; . . . , 1,. . .)T, with integer and non-negative coefficients aikr, identifying a valid packing pattern assigned to a bin with capacity Wk, i.e., a combination of aikr items of size wi, P i = 1, . . . , m,  Pm  K that fits in this bin a w 6 W admissible patterns per bin class k and, so, . There are p ikr i k k i¼1 k¼1 p k ¼ p. Variables xkr are general integer variables. They represent the number of times a pattern r associated to a bin with capacity Wk is used. Inequalities (2) model the demand constraints, whereas (3) ensure that the number of available bins is not exceeded. Inequalities (2) might be replaced by equalities. Gilmore and Gomory [14] showed that an optimal solution always exists with null slack variables in (2). Nevertheless, we will keep these inequalities since they lead to a more restricted dual solution space. In this paper, we consider the resolution by column generation of the LP relaxation of (1)–(4), which will be denoted by P. Let ui, i = 1, . . . , m, and vk, k = 1, . . . , K, be the dual variables associated to constraints (2) and (3), respectively. For ease of presentation, we will frequently refer to the ui and vk variables, respectively, as the items’ and bins’ dual variables. Based on the complementary slackness conditions, we can give an interesting interpretation to the items’ dual variables. If the availability of the bins was unbounded, the ui variables would represent the exact ideal sizes the items should have in order to fulfill the bins selected in the primal optimal solution. From the dual standpoint, a pattern that is in the optimal primal basis with a positive value corresponds to a constraint with a slack variable that is equal to 0. Instead of their original sizes, if the items had sizes equal to the corresponding items’ dual variables, the patterns that are in this optimal primal basis would become patterns without any unused space. In the bounded case, when the availability constraints are effective, the vk variables relax the dual knapsack constraints to some point. The dual formulation D of P follows: ðDÞ

max subject to

m X i¼1 m X

ui bi þ

K X

ð5Þ

v k Bk ;

k¼1

aikr ui þ vk 6 W k ;

k ¼ 1; . . . ; K; r ¼ 1; . . . ; pk ;

ð6Þ

i¼1

ui P 0; vk 6 0;

i ¼ 1; . . . ; m; k ¼ 1; . . . ; K:

ð7Þ ð8Þ

The LP relaxation of (1)–(4) can be obtained through an adequate Dantzig–Wolfe decomposition of two other LP models: the extended Kantorovich model [20], and the arc-flow model described in [29,30]. These models are usually called compact or original models [33]. The former relies on assignment variables, and is known for its poor LP bound. It exhibits a high degree of symmetry, which is characterized by many alternative solutions corresponding in fact to the same packing. The LP lower bound of the arc-flow model is as strong as the one provided by the LP relaxation of (1)–(4), but the high number of constraints of this model may penalize its usability. Formulation (1)–(4) is in fact the most popular formulation for the class of problem we are studying in this paper. Gilmore and Gomory [15] used it to model their machine balance problem, but it can also be used to model the multiple length cutting stock problem, and the variable sized bin-packing problem as well. 3. Dual-optimal inequalities In [31], Vale´rio de Carvalho introduced a certain concept of dual cuts, columns in the primal that do not affect the optimal value as long as a solution to the original problem can be recovered at no cost with those

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1337

columns at the zero level. Later on, Ben Amor et al. [5] used the term dual-optimal inequalities, and distinguished between weak and deep inequalities. In the former, no optimal solution of the original formulation is excluded, whereas the deep inequalities may cause the rejection of some alternative optimal solutions. The cuts presented in [31] apply to the variable sized bin-packing problem. We recall them briefly in Section 3.1. In Section 3.2, we introduce new types of weak and deep dual-optimal inequalities specially devised for the variable sized bin-packing problem. 3.1. Inequalities on items’ dual variables If we observe the values of an optimal dual solution to (5)–(8), the ordering of the items’ dual variables will be quite evident. After all, the items’ dual variables are subject to constraints whose coefficients are directly related to the set of item sizes. This conduces to an ordering P that is completely dependent on the size of the items. Let S be the set of items of size ws, such that s2S ws 6 wi , for some i = 1, . . . , m. The relation can be expressed as follows: X us 6 0; i ¼ 1; . . . ; m; 8S: ð9Þ ui þ s2S

Depending on the cardinality of S, different types of dual inequalities may be defined. The result is an exponential number of valid constraints. Adding these inequalities to D leads to a so-called extended formulation. In the primal, the respective columns allow an item in a pattern to be exchanged by a combination of other items with smaller or equal total size. Vale´rio de Carvalho showed that any optimal solution to the non-extended formulation verify inequalities (9), which make them weak dual-optimal inequalities. These results remain even in the case of the variable sized bin-packing problem. Extensions to the proofs are straightforward and, so, we omit them. During the successive resolutions of the restricted master problems in a column generation process, since the columns are not completely enumerated, these inequalities are frequently violated. The inclusion of a bounded number of such cuts is opportune. Furthermore, when we add inequalities (9) to the dual, it may happen that some basic dual solutions associated to a given degenerate extreme point in the primal are cut. As a consequence, adding the corresponding extra columns to the primal may also reduce the primal degeneracy [31]. 3.2. Inequalities on bins’ dual variables Let De (Pe) correspond to D (P) extended with additional inequalities on the vk dual variables (primal columns, respectively) ðP e Þ

min

ðDe Þ

cx þ fy;

max

ub þ vB;

Ax P b; Ex þ Fy 6 B;

uA þ vE 6 c; vF 6 f ;

x; y P 0:

u P 0; v 6 0:

A family of dual inequalities can be stated using the following argument: a packing pattern associated to some bin can always be reassigned to another bigger bin at a cost that equals the difference between the two capacities. In the primal, this operation is allowed by including columns with a +1 in row m + k, a 1 in row m + k 0 , and a cost of W k  W k0 , k = 1, . . . , K  1, k 0 = 2, . . . , K, and W k > W k0 . In the dual, we will be inserting cuts with the form: vk  vk 0 6 W k  W k 0 ;

k ¼ 1; . . . ; K  1; k 0 ¼ 2; . . . ; K; W k > W k0 :

ð10Þ

Let ð~x; ~y Þ be a valid solution for Pe. In the case where F ~y 6¼ 0, ~x may be infeasible for P, as illustrated in Example 3.1. The next proposition, and the corollary that follows, show that the LP lower bounds of P and Pe remain the same.

1338

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

Proposition 3.1. Given a feasible solution ð~x; ~y Þ to Pe, we can always recover a feasible solution to P with an equal or lower cost. Proof. Let AE and NF be the subset of columns in Pe associated to the patterns and to the dual inequalities (10), respectively. Starting with ð~x; ~y Þ, the following step-by-step procedure shows how to build a solution expressed only as a non-negative combination of feasible packing patterns. To clarify the presentation, the k index was dropped from the columns (a1kr, a2kr, . . . , amkr; e1kr, . . . , eKkr)T of Pe. for k :¼ K, . . ., 2 Let AE be the subset of columns (a1r, a2r, . . . , amr; e1r, . . . , eKr)T of AE with ~xr > 0, and an unit coefficient in row m + k (ekr = 1). Let NF be the subset of columns NFj = (0, . . . , 0; . . . , 1, . . . , 1, . . .)T from NF with ~y j > 0, a cost of fj units, the 1 occurring at position m + k and the +1 at position sj. for all j : NFj 2 NF P while ~y j > 0 and r:AEr 2AE ~xr > Bk T Select a column ða1r ; a2r ; . . . ; amr ; . . . ; esj r ; . . . ; ekr ; . . . Þ from AE with ~xr > 0 T AEnew ¼ ða1r ; a2r ; . . . ; amr ; . . . ; esj r þ 1; . . . ; ekr  1; . . . Þ . This column has a cost of Wk + fj; xnew is the associated primal variable. if ð~y j > ~xr Þ ~y j :¼ ~y j  ~xr , xnew :¼ ~xr and ~xr :¼ 0 else ~xr :¼ ~xr  ~y j , xnew :¼ ~y j and ~y j :¼ 0 end if Add AEnew to Pe and update the solution with xnew end while if ~y j > 0, ~y j :¼ 0 end for end for The final solution is valid for Pe and have all the columns referring to the dual inequalities (10) at the zero level. Therefore, we get a feasible solution for P. If condition ~y j > 0 at the end of the algorithm is true at least once, the resulting solution will have a cost that is lower than the one associated with the original solution ð~x; ~y Þ. Otherwise, the cost remains the same. h Corollary 3.1. Since Pe is a relaxation of P, the optimal solutions of P and Pe have the same cost. We prove next that inequalities (10) are in fact weak dual-optimal inequalities. Proposition 3.2. Any optimal dual solution (u*, v*) to D satisfies inequalities (10). Proof. Consider an optimal primal-dual solution pair x* and (u*, v*) to P and D, respectively. Using arguments based on the complementary slackness conditions, we will show that this dual optimal solution obeys to inequalities (10). For this purpose, we consider each bin type separately, and start with a bin type k. In the optimal primal solution x*, two cases may occur: (1) at least one pattern associated to bin k is in the  optimal  ðxkr > 0, for some r), or (2) no pattern associated to bin k is in the optimal basis Ppk  basis r¼1 xkr ¼ 0 . For the case (1), let Akr = (a1kr, a2kr, . . . , amkr)T, and Ekr = (. . . , 1 , . . .)T, the +1 occurring at position k, be the elements of a column in P with a positive value in x*. This column has a null reduced cost, i.e., Wk  u* Æ Akr  v* Æ Ekr = 0. If we assign the items of the corresponding pattern to a bigger bin, say k 0 , we get another valid pattern; let Ak0 r0 and Ek0 r0 be the elements of the corresponding column. That new pattern may be in the optimal basis, or not, but its reduced cost will always be non-negative, i.e., W k0  u  Ak0 r0  v  Ek0 r0 P 0. Subtracting the reduced costs, we get W k0  W k  u  ðAk0 r0  Akr Þ  v  ðEk0 r0  Ekr Þ ¼ W k0  W k  vk0 þ vk P 0. Thus, vk0  vk 6 W k0  W k . This inequality holds no matter what bin k 0 is considered, as long

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1339

as W k0 > W k . This means that, if there is in x* a pattern associated to bin type k with a positive value, all the inequalities (10) relating k P with other bigger bins will be satisfied. pk For the case (2), since r¼1 xkr ¼ 0, the slack associated to the availability constraint of bin k will be positive, and, by the complementary slackness conditions, vk is equal to 0. Since the v variables in D are nonpositive, for all the k 0 such that W k0 > W k , inequality vk0  vk 6 W k0  W k still holds. The point here is that, whatever the structure of x* concerning the inclusion or not of patterns associated to a particular bin type k, we are always able to show that inequalities (10) relating k with other bigger bins are satisfied. The same reasoning applies for any bin type k, and hence, all the inequalities (10) are satisfied by (u*, v*). h Example 3.1. Consider an instance with a set W of bins, W = (7, 7, 5, 4, 4, 4), and a set w of items, w = (3, 3, 3, 2, 2, 2, 1, 1). A possible restricted master problem comprising the set of linearly independent dual inequalities associated to (10) ð~y 1 ; ~y 2 Þ is illustrated in Fig. 1. A solution ð~x; ~y Þ with ~x1 , ~x3 , ~x4 and ~y 1 equal to one, and the other variables equal to zero, is feasible for the extended model. However, ~x is infeasible for the non-extended model since the limit on bins with capacity 5 is exceeded. Additionally, two valid dual inequalities were derived that strengthen D even more. In the sequel, e is a small positive value. The first dual inequality is as follows: vK P W K  W 1  e:

ð11Þ

0

Proposition 3.3. Let ð~x ; y  Þ be an optimal solution to P e , the extended version of P comprising the additional primal decision variable y* associated to the dual inequality (11). If P is feasible, then y* = 0, and ~x is optimal for P. Proof. First, note that, in the primal, the column associated to (11) has a positive cost equal W1  WK + e. Pto pK  * is positive. Hence, we must have ~xKr > BK , and Our proof will be made by contradiction. Assume that y r¼1 Pp K  ~xKr  BK . If this was not the case, clearly, a solution would exist with a lower cost than ð~x ; y  Þ, cony  ¼ r¼1 tradicting the optimality of ð~x ; y  Þ. Furthermore, for ð~x ; y  Þ, the sum of the slack variables of the availability   * constraints must be greater P than or equal to y . Again, if this was not the case, this would imply that ð~x ; y Þ has a cost greater than k W k Bk contradicting the feasibility of P. In practice, having y* > 0 means that we are using extra units of the bin type K, which are not available. According to the complementary slackness conditions, we have vK ¼ W 1  W K þ e. As mentioned, there will be a bin, larger than the smallest, with positive slack. Without loss of generality, assume that this positive slack occurs in the biggest bin (the results holds for any other bin larger than the smallest). This implies that v1 ¼ 0. Let Ptr = (a1Kr, a2Kr, . . . ,P amKr; 0, 0, . . . , 1) be one of the patterns associated m    ~ to the bin type K with a positive value in x , i.e., such that i¼1 ui aiKr þ vK ¼ W K . Such a pattern must exist. Pm  0 The equality i¼1 ui aiKr ¼ W 1 þ e follows. A pattern, say r , associated to the biggest bin, and with the same set of items as Ptr, is a valid pattern. P This pattern has the form Ptr0 ¼ ða1Kr ; a2Kr ; . . . ; amKr ; 1; 0; . . . ; 0Þ. Its   reduced cost, which is given by W 1  m i¼1 ui aiKr þ v1 , is equal to e. This contradicts the optimality of   ð~x ; y Þ. h

Fig. 1. Restricted master problem (Example 3.1).

1340

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

The second dual inequality restricts the set of possible values for the v1 dual variable, i.e., the one related with the biggest bin. This inequality is valid for instances with items smaller than the smallest bin, and is stated as follows:   W1 v1 P W 1  ð12Þ W 1  e: wm 0

Proposition 3.4. Let ð~x ; y  Þ be an optimal solution to P e , the extended version of P comprising the additional primal decision variable y* associated to the dual inequality (12). If P is feasible, then y* = 0, and ~x is optimal for P. Pp 1  Proof. This proof is made by contradiction. Assume that y* > 0. This implies that xl 1r >mB1 and r¼1 ~ Pp 1    y ¼ r¼1~x1r  B1 . By the complementary slackness conditions, we have also v1 ¼ W 1 þ Wwm1 W 1 þ e, Pm and i¼1 ai1r ui þ v1 ¼ W 1 , for at least l mone pattern r associated to the bin of capacity W1. Combining both Pm  the equations we get i¼1 ai1r ui ¼ Wwm1 W 1 þ e. At least one constraint on the availability of the bins must have a corresponding slack P variable that is 0 positive. Indeed, since P is feasible and P el is ma relaxation of P, the following holds: zP e0 6 Kk¼1 W k Bk . The P Ppk  PK by W 1 B1 þ y  Wwm1 W 1 þ y  e þ Kk¼2 r¼1 xkr W k . The inequality value of zP e0 is given k¼2 W k Bk  l m PK Ppk   W1  k¼2 r¼1 xkr W k P y wm W 1 þ y e follows, indicating that there is a positive slack in at least one availability constraint. Since all the items fit in the smallest bin, all the patterns that consist in a single item i assigned to a bin k are feasible, whatever the bin k. The corresponding dual constraints ui þ vk 6 W k , i = 1, . . . , m, k = 1, . . . , K, are also valid for the dual problem. Since there is a positive slack variable in at least one availability constraint, say k 0 , by the complementary slackness conditions, we have vk0 ¼ 0, and ui 6 W k0 6 W 1 , for i = 1, . . . , m. Furthermore, given that wm is the sizelof mthe smallest item, whatever the pattern (a1kr, a2kr, . . . , amkr; . . .), the Pm W1 following inequalityl holds: W1, we i¼1 aikr 6 wm . Hence, for any pattern r associated to a bin with lcapacity m m Pm Pm W1 W1   have i¼1 ai1r ui 6 wm W 1 , which contradicts the conclusion stated above that i¼1 ai1r ui ¼ wm W 1 þ e. h With the inequalities (11) and (12), the dual variables are confined to a fixed and non-empty box, which can be defined before starting column generation. This box contains at least an optimal dual solution. In the literature, there are many evidences pointing to the efficiency of column generation when this kind of boxes are used [4]. The following example shows that inequalities (11) and (12) are deep dual-optimal inequalities. Example 3.2. Consider an instance similar to the one presented in Example 3.1, except for the demand vector P b = (5, 6, 4). In the optimal solution, all the bins are used, because m i¼1 wi bi ¼ 5  3 þ 6  2 þ 4 ¼ 31. An optimal primal solution is x1 = 2, x4 = 1, x5 = 1 and x6 = 2. An optimal dual solution is u1 = 24, u2 = 16, u3 = 8, v1 = 49, v2 = 35 and v3 = 28, showing that v3 P 3  e and v1 P 42  e, inequalities (11) and (12), respectively, for this instance, are deep dual-optimal inequalities. The alternative optimal dual solution u1 = 3, u2 = 2, u3 = 1, v1 = v2 = v3 = 0 is not cut by the deep dual-optimal inequalities. Moving all the items from a bin to another empty bin leads to an increase of the unused space equal to the difference between the bin capacities, which, if it is big enough, may be fulfilled with other items. However, this operation does not translate into valid dual inequalities. In fact, if we consider adding items’ dual variables to (10), and if we add the corresponding columns to P, the result will be a true relaxation of P with a poorer lower bound. This is illustrated by the following trivial example. Example 3.3. Assume that two items of size 5 and 2, respectively, have to be packed into bins with capacities 5 and 3, and suppose that 2 bins of each type are available. The feasible packing patterns can be denoted as follows: (1, 0; 1, 0)T, (0, 1; 1, 0)T and (0, 1; 0, 1)T. For example, the second pattern means that an item with a size equal to 2 is packed in a bin of capacity 5. The LP optimal solution of this problem has a cost equal to 8, and consists in using an unit of the first pattern, and another unit of the last pattern. By adding the extra column

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1341

(0, 1; 1,  1)T to the primal with a cost equal to 2 (the difference between the two capacities), the LP optimal solution cost reduces to 7 (an unit of the first pattern, plus an unit of the extra column). In the primal, this extra column means that a pattern related to a bin with capacity 3 can be transferred to a bin with capacity 5 at a cost equal to 2. With this transfer, the unused space in the pattern is at least equal to 2 units, which is enough to accommodate the smallest item of the instance. However, translating this operation into a single primal column, and adding it to the LP master leads to a relaxation of the original LP. 4. Alternative aggregation schemes In the past, aggregation and disaggregation techniques were essentially used to deal with problems coming from memory space restrictions. These difficulties were becoming even more constraining as researchers were trying to solve problems with a growing level of precision. By concentrating on a restricted but sufficiently representative set of data, one will also expect to see the computational burden reduced. Rogers et al. [27] gave a comprehensive survey on the contributions made in the field, and synthesized the different elements of an aggregation/disaggregation process. In this section, we describe two alternative aggregation schemes. The goal is to accelerate the search for an optimal solution through column generation by solving sequences of easier approximations, i.e., aggregated problems. Here, aggregation is seen as an implicit form of controlling the dual variables. The methods rely exclusively on the aggregation of items and, therefore, they can be applied to the single sized bin-packing problem. In Section 4.1, we present a simple two-phase column generation algorithm that starts by solving a row aggregated model, which corresponds to a problem with larger items. In Section 4.2, an iterative algorithm is presented that takes advantage of the phenomenon of identical prices, soon pointed out by Gilmore and Gomory [15], using smaller LPs where both variables and constraints are aggregated. The resulting doubly aggregated models give a good approximation to the disaggregated problems. 4.1. A simple row aggregation scheme In a variable sized bin-packing problem, when the number of items that can fit in the bins is high, the LP matrix has an high density, and this favors the occurrence of primal degeneracy. If we were able to anticipate how some of the items are combined in the optimal solution, larger items could be defined, and density could be reduced. Unfortunately, it is difficult to find the right combinations and, so, we will approach this problem heuristically. We solve the variable sized bin-packing problem by column generation in two steps. In the first, the items of the original problem are combined pairwise, leading to an approximation Pra of P. Since we are restricting the original solution space, we get an upper bound on the optimal solution of P; its quality will be as good as will our guess for the combination of the items. Note that the quality of the approximation will certainly tend to decrease if greater combinations are tried. In the second step, we guarantee the convergence to the optimal solution of P by solving P starting with the columns of the last restricted master problem relative to Pra, properly disaggregated. We use the simple scheme RA to aggregate the items of the original problem. Aggregation scheme RA Let wra and bra be, respectively, the set of item sizes and demands of the aggregated problem. Initially, ra w = ; and bra = ;. Let b, indexed by i, represent the set of demands of the original problem, and q be the number of positive values in b. i :¼ 1 while i 6 m and q P 2 while bi > 0 and q P 2 Let kj be the index of the smallest bin where the item i and another item, say j, fit together, i.e., wi þ wj 6 W kj .

1342

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

Let j be the index of an item of the original problem between i + 1 and m, with size wj, and bj > 0, such that the difference between wi + wj and W kj is minimized. if (bj P bi) bj :¼ bj  bi, bra new :¼ bi and bi :¼ 0 else bi :¼ bi  bj, bra new :¼ bj and bj :¼ 0 end if ra Add size wi + wj to wra with a demand of bra new units if it does not already exist or increment by bnew its demand, otherwise. i :¼ i + 1 end while end while if bi > 0, add wi to wra with a demand of bi units if it does not already exist or increment by bi its demand, otherwise. Pjwra j Clearly, the total number of items ð i¼1 bra i Þ decreases, while the number of different types of items in the aggregated problem will only be smaller compared to the original one if equality bj = bi holds, or if bi is equal to 0 in the last step. Since the combination of items with the same size is not allowed, the number of different types of items will never increase. An item is combined with a smaller one if the difference between the sum of their sizes and the capacity of the smaller bin where this combination fits is the minimum over all the possible combinations that include the first item. With such a criterion, if an item i goes with an item j, item i + 1 will frequently be combined with the item j  1, particularly if wi + wj is almost equal to wi+1 + wj1. This type of association is frequent in optimal solutions, in which some of the patterns differ only in a small number of items. The unused space within a pattern is used to cope with the small difference between pairs of items. These pairs finally appear in very similar patterns. Another interesting point is the fact that the dual inequalities of Section 3.1 are particularly efficient in terms of stabilization for instances with groups of almost identical items. The rows of P are subject to an equivalent aggregation process. The following example illustrates the steps followed by the aggregation scheme. Example 4.1. Consider an instance with a set of item sizes w = (90, 59, 58, 57, 25, 20) and demands b = (5, 5, 7, 3, 7, 1). Ten bins with capacities 160, 120 and 100 are available. Fig. 2 illustrates a possible restricted master problem. The aggregation proceeds as follows. We start with wra = ; and bra = ;. Five units of w1 = 90 are initially aggregated to 5 other units of w5=25, leading to wra = (115) and bra = (5). The 5 units left in W2 = 120, the smaller bin where an item of size 115 fits, are the smaller unused space over all the other combinations that include w1. Subsequently, 5 items of size 59 are combined with 5 items of size 58 (wra = (117, 115), bra = (5, 5)), 2 items of size 58 with 2 items of size 57 (wra = (127, 115), bra = (5, 7)), 1 item of size 57 with 1 item of size 25 (wra = (117, 115, 82), bra = (5, 7, 1)), and finally 1 item of size 25 with 1 item of size 20 (wra = (117, 115, 82, 45), bra = (5, 7, 1, 1)). The aggregated problem has four different types of items for a total demand of 14 units, corresponding to half the 28 items of the original problem.

Fig. 2. Restricted master problem (Example 4.1).

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1343

Fig. 3. Row aggregated LP (Example 4.1).

The aggregation of the items leads to a similar row aggregation in P. Fig. 3 shows the result for the restricted master problem of Fig. 2. A column in P will have an associated column in the aggregated model only if all the items of the corresponding pattern can be combined among them according to the item aggregation defined by RA. Thus, the first column in Fig. 2 will have a counterpart in the aggregated LP, while the second will not. For the latter, the unique item of size 90 combines with the item of size 25 and the remaining item of size 25 goes with the item of size 20. The disaggregation process is straightforward. Note that a column of the aggregated formulation may generate one or more columns for the disaggregated problem. Besides the sparser density of the aggregated problem, which reduces its degeneracy, this problem is also easier to solve for other obvious reasons. The set of valid packing patterns is smaller compared to the original problem. Fewer items are needed to fulfill the bins. Their demands are also lower. As a consequence, the column generation subproblems are solved faster. Moreover, the disaggregation cost is not high. The efficacy of the algorithm depends on how accurate is the combination of items. We are looking for a good set of initial columns to start the resolution of the original problem by column generation, and we expect that the effort to find them will be essentially concentrated in the first phase where a simpler and less degenerated problem is solved. The computational experiments presented in Section 5 show that for some instances this algorithm brings some appreciable improvements. 4.2. Implicit dual constraints: A double aggregation scheme The optimal solutions of bin-packing problems have an interesting characteristic: items that are almost identical have dual variables that are frequently equal. Gilmore and Gomory [15] already experienced this phenomenon of identical prices in various stages of the column generation algorithm, for low and high waste problems. For the cutting stock problem, they give an explanation based on the fact that the trim loss in some cutting patterns belonging to the basis allows some of their items to be interchanged. In formulations in which a demand constraint is defined for each single item, there exists an optimal solution with equal dual variables for the items of the same size. This result is well known. Ben Amor et al. [5] report computational results where it is shown that convergence is improved when a unique constraint per item size is considered. What happens is that the dual variables of identical items are forced to be equal. In this section, we propose an algorithm that solves successive approximations obtained by imposing these equality constraints to items with different sizes. From the problem definition standpoint, items with nearly the same size are replaced by a single item with a size equal to the average size of the original items. This operation amounts to aggregating columns followed by a constraint aggregation. A column of the aggregated model will correspond to a set of columns with more rows in the disaggregated model. The following example illustrates this correspondence. Example 4.2. Remember the problem of Example 4.1. The patterns associated to x9, x10 and x11 (respectively (0, 1, 1, 0, 0, 0; 0, 1, 0)T, (0, 1, 0, 1, 0, 0; 0, 1, 0)T and (0, 0, 1, 1, 0, 0; 0, 1, 0)T) differ in a single item. There are other patterns that differ from the previous in a single item such as (0, 2, 0, 0, 0, 0; 0, 1, 0)T, (0, 0, 2, 0, 0, 0; 0, 1, 0)T and (0, 0, 0, 2, 0, 0; 0, 1, 0)T. If we replace the items of size 59, 58 and 57 by an item with size 58 (the average of 59, 58 and 57), and a demand of 15 units (the sum of the corresponding original demands), the pattern (0, 2, 0, 0; 0, 1, 0)T in the aggregated model will now stand for all the aforementioned patterns.

1344

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

The aggregation scheme is now described more formally. We define an equivalence group Gl as the set of successive items i, i + 1, . . . , i + jGlj such that wi  wi+1 6 D. Let mda be the total number of equivalence groups in the original problem for some positive value D. We have mda 6 m. Two patterns AE1 and AE2 are said to be adjacent if they are associated to the same bin type, and if AE1 is obtained from AE2, and vice versa, by replacing some items of AE2 each one with an item belonging to the same equivalence group. The columns of P, the original formulation, are divided in groups of adjacent patterns, say AEr , r ¼ 1; . . . ; p. The aggregated model Pda results from the aggregation of columns and rows within each set AEr defined by the following linear operations: T  AEr  S r ; where T is a mda · m matrix with coefficients equal to 1 and a block diagonal structure, and Sr is a j AEr j 1 column vector, r ¼ 1; . . . ;  p. There is a unit coefficient in row l and column i of T iff i 2 Gl. All the elements of Sr are equal to 1=jAEr j. The question now is which size should have the mda items of the aggregated problem, one per each original equivalence group, such that all the columns in Pda (and no others) are feasible patterns. According to the aggregation scheme, these item sizes depend on each column of Pda and, in particular, on the original set of adjacent columns AEr in P that originate them. A way to have a unique size for each of the mda items is to make the equivalence groups coincide with the set of dual variables that are equal in an optimal dual solution, and to set the sizes equal to the optimal dual values accordingly. However, an optimal dual solution is rarely available beforehand. Searching for a set of sizes such that all the columns in Pda are feasible starting with a set of predefined equivalence groups is not guaranteed to succeed. In fact, such sizes may not exist. The alternative consists in using approximate sizes. Setting the size of the ith item, i = 1, . . . , mda, equal to the size of the smallest item in Gi leads to an aggregated problem that is a relaxation of P. On the other hand, if we choose the size of the largest item in Gi, Pda will be a restriction of P. We chose to use the first integer value lower than the average size of the items in Gi, i = 1, . . . , mda. The consequence is that some columns of P may not have a representation in the resulting Pda. Example 4.3. Consider an instance with a set of items w = (80, 70, 60, 55, 43, 38, 33, 20, 16) to be packed in bins of capacity 100. Fig. 4 shows the complete set of feasible patterns for this instance. With D = 5, five equivalence groups can be defined: G1 = (80), G2 = (70), G3 = (60, 55), G4 = (43, 38, 33) and G5 = (20, 16). They lead to the seven different sets of adjacent columns identified in Fig. 4. Figs. 5 and 6 illustrate the result of the successive column and row aggregation, respectively. The new set of items for the aggregated problem is wda = (80, 70, 57, 38, 18). For the pattern (0, 0, 1,1, 0)T, which replaces the columns of AE1 , the size of the item that stands for those of G4 should be 37 ((1/5 · 43 + 2/5 · 38 + 2/5 · 33)/ (1/5 + 2/5 + 2/5)), while for the pattern (0, 0, 0, 2,1)T this size should be 36.6 ((3/9 · 43 + 7/9 · 38 + 8/9 · 33)/ 2). By using the averages for each equivalence group, pattern (0, 0, 0, 0, 0, 0, 3, 0, 0)T can no longer be represented in the aggregated model (3 · 38 > 100) and the aggregation of (0, 0, 0, 0, 2, 0, 0, 0, 0)T results in a pattern (0, 0, 0, 2, 0)T that has enough waste to cope with another item. Note that it is convenient for the items of an equivalence group to appear in quite similar patterns. This situation is likely to appear when the item sizes are relatively near. Suppose, for example, that we include the items of size 80 and 70 in the same equivalence group. This can be done with D = 10. In this case, there are three equivalence groups: G1

Fig. 4. Set of feasible patterns (Example 4.3).

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1345

Fig. 5. Set of patterns after column aggregation (Example 4.3).

Fig. 6. Final set of patterns (Example 4.3).

comprises the items with sizes 80, 70, 60 and 55, G2 the items with sizes 43, 38 and 33, and G3 the items with sizes 20 and 16. The items of these equivalence groups are replaced by items with sizes 66, 38 and 18 (the average of the original sizes). Items of size 66 cannot combine with items of size 38 and hence, this choice will increase the number of patterns that cannot be represented in the aggregated model. This aggregation can be done repeatedly. A problem P can be aggregated using D = D1, leading to an aggreda gated problem P da 1 that can in turn be aggregated with D = D2, and so on. Disaggregation is applied to P i to da recover the set of original adjacent columns of P i1 , or P if i = 1. da To solve P, we use a n-phase algorithm that solves the sequence P da n1 , P n2 ; . . . ; P by column generation. A limit is imposed in the number of items in each equivalence group. Ideally, this control should be made through the parameters Di. The columns in the final restricted master problem of P da i are partially disaggregated. For each column that is not in the basis, only one column of the original set of adjacent columns is generated. For those that are in the basis, we generate a subset of original columns keeping the coefficients of the aggregated column. Therefore, if pattern (0, 0, 0, 2,1)T in Example 4.3 is in the optimal solution, only (0, 0, 0, 0, 0, 2, 0, 1, 0)T, (0, 0, 0, 0, 0, 0, 2,1, 0)T, (0, 0, 0, 0, 0, 2, 0, 0, 1)T and (0, 0, 0, 0, 0, 0, 2, 0, 1)T will be generated. The resolution of P da i1 starts with this initial set of disaggregated columns. An artificial column with a high cost guarantees the feasibility of the disaggregated master problem. The nth iteration consists in solving the original problem P starting with the disaggregated columns of P da 1 . 5. Computational experiments We report the results of three groups of tests performed on randomly generated instances and other well known instances from the literature. We compare the effectiveness of the different approaches based on the number of column generation iterations and the total computing time. A starting set of columns was computed through a FFD heuristic, where bins were filled in order of increasing capacities. An artificial column was added to the restricted master problem in case the heuristic does not provide a valid solution due to the availability constraints on the bins. The knapsack subproblems were solved using the mt1r procedure of Martello and Toth [22]. At most one column per bin class was generated in each iteration. Our computational experiments were conducted on a 700 MHz Pentium III with 128 Mb of RAM under Windows ME operating system. The algorithms were coded using C++ and CPLEX 6.5 [18]. Table 1 illustrates the relative performance of the dual inequalities introduced in Section 3. Comparison is made between the standard column generation algorithm, column generation with the additional dual

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1346

Table 1 Performance of the inequalities on items’ and bins’ dual variables m 168.0

102.6

128.0

152.8

177.9

179.5

k 5.0

11.0

11.0

11.0

11.0

15.0

Strategy

splp

% red

tlp

% red

Standard column generation Inequalities on items’ dual variables Inequalities on items’ and bins’ dual variables

162.6 132.1 80.0

18.8 50.8

16.9 14.7 8.7

13.0 48.5

Standard column generation Inequalities on items’ dual variables Inequalities on items’ and bins’ dual variables

50.8 49.5 35.4

2.6 30.3

3.2 3.5 2.4

9.4 25.0

Standard column generation Inequalities on items’ dual variables Inequalities on items’ and bins’ dual variables

72.3 64.2 45.5

11.2 37.1

7.6 7.4 4.8

2.6 36.8

Standard column generation Inequalities on items’ dual variables Inequalities on items’ and bins’ dual variables

85.9 78.6 54.6

8.5 36.4

13.2 13.6 8.2

3.0 37.9

Standard column generation Inequalities on items’ dual variables Inequalities on items’ and bins’ dual variables

108.0 98.1 64.2

9.2 40.6

25.2 25.4 14.8

0.8 41.3

Standard column generation Inequalities on items’ dual variables Inequalities on items’ and bins’ dual variables

87.0 84.1 62.7

3.3 27.9

24.7 26.4 18.3

6.9 25.9

inequalities (9) and column generation considering inequalities (9)–(12). Dual inequalities were added prior to the first resolution of the LP relaxation with a small perturbation of their right hand side for (9) and (10) so that the respective columns are at the zero level in the final solution. To limit the number of columns inserted, the maximum cardinality of S has been set to jSj = 2. Only one column per item was considered when jSj = 2. The tests were carried out on a set of randomly generated problems inspired in the triplets instances from the OR-library [2]. For these instances, the solution consists in bins receiving exactly three items. The only difference between the triplets instances of the OR-library and our test-problems is that a limited number of bins with various capacities are now considered. Six groups of 50 instances each were generated with m ranging from 100 to 180 and K from 5 to 15. Items have integer sizes between 100 and 360 while the capacities of the bins vary between 400 and 720. Table 1 shows the average computing time in seconds (tlp) and the average number of column generation iterations (splp) obtained with the three alternative strategies. A percentage of reduction (% red) in time and iterations is indicated comparing the standard column generation algorithm to the other two methods. The inequalities on items’ dual variables seem to have only a limited impact on those instances. The set of inequalities imposed on the bins’ dual variables appears as a good complement to reach an interesting level of stabilization. For the first group of instances, which is also the harder with an average number of iterations of 162.6, the combined application of these dual inequalities yields a reduction of approximately 50% in the computing time and number of subproblems solved. Note that the instances were generated such that the constraints on bins’ availability were truly effective. Indeed, dual inequalities (10)–(12) will be as strong as will those restrictions. In practice, we will surely found many problems with this characteristic. In Table 2, we report the average computational results obtained for a subset of the instances used in a recent publication of Belov and Scheithauer [3]. These results illustrate the impact of the two-phase algorithm based on the aggregation scheme RA of Section 4.1 (procedure RA, for short) compared to the standard column generation algorithm with and without the valid dual inequalities discussed in this paper. We chose 15 groups of 50 instances each, with m = 100 and m = 150 and a number of bin types varying between 2 and 10. With procedure RA, the reduction in the number of column generation iterations achieved with dual inequalities is almost always duplicated. For one group of instances (m = 99.40 and K = 3.80), the improvement reaches 36.2%. The computing time does not generally decrease in the same proportion. However, there

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1347

Table 2 Performance of aggregation scheme RA m 99.10

99.08

99.08

99.08

99.08

99.03

99.10

99.10

99.08

99.31

99.40

99.23

148.50

148.47

99.09

k

Strategy

splp

% red

tlp

3.70

Standard column generation Dual inequalities Procedure RA + dual inequalities

135.3 110.9 86.1

18.0 36.4

11.7 5.4 5.1

53.8 56.4

Standard column generation Dual inequalities Procedure RA + dual inequalities

121.0 100.9 82.2

16.6 32.1

8.6 5.3 6.1

38.4 29.1

Standard column generation Dual inequalities Procedure RA + dual inequalities

102.3 84.3 68.6

17.6 32.9

10.4 4.7 5.1

54.8 51.0

Standard column generation Dual inequalities Procedure RA + dual inequalities

121.3 99.0 78.9

18.4 35.0

5.1 5.3 5.6

3.9 9.8

Standard column generation Dual inequalities Procedure RA + dual inequalities

119.8 99.7 80.8

16.8 32.6

8.8 5.3 5.5

39.8 37.5

Standard column generation Dual inequalities Procedure RA + dual inequalities

136.9 106.3 80.6

22.4 41.1

7.3 4.4 3.4

39.7 53.4

Standard column generation Dual inequalities Procedure RA + dual inequalities

128.1 101.4 81.7

20.8 36.2

2.4 2.7 2.2

12.5 8.3

Standard column generation Dual inequalities Procedure RA + dual inequalities

125.2 100.4 80.6

19.8 35.6

3.6 3.9 3.4

8.3 5.6

Standard column generation Dual inequalities Procedure RA + dual inequalities

123.1 100.7 79.8

18.2 35.2

4.3 5.5 4.5

27.9 4.7

Standard column generation Dual inequalities Procedure RA + dual inequalities

98.8 74.7 44.6

24.4 54.9

6.3 2.4 1.8

61.9 71.4

Standard column generation Dual inequalities Procedure RA + dual inequalities

81.7 60.5 31.0

25.9 62.1

1.4 1.3 0.7

7.1 50.0

Standard column generation Dual inequalities Procedure RA + dual inequalities

104.0 84.0 64.2

19.2 38.3

6.8 7.0 5.6

2.9 17.6

Standard column generation Dual inequalities Procedure RA + dual inequalities

182.8 144.4 110.7

21.0 39.4

17.2 15.5 13.8

9.9 19.8

Standard column generation Dual inequalities Procedure RA + dual inequalities

220.6 176.0 141.4

20.2 35.9

46.0 20.5 18.9

55.4 58.9

Standard column generation Dual inequalities Procedure RA + dual inequalities

85.0 71.2 59.8

16.2 29.6

8.4 7.8 7.7

7.1 8.3

4.00

3.98

3.73

4.00

3.87

3.74

3.74

3.78

3.77

3.80

6.23

3.74

3.77

8.67

% red

1348

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

Table 3 Computational results for the best 10 instances of each group in Table 2 m 99.10

99.00

99.00

99.00

99.00

99.00

99.00

99.00

98.90

99.20

99.30

99.10

148.40

148.50

99.30

k

Strategy

splp

3.50

Standard column generation Dual inequalities Procedure RA + dual inequalities

104.4 85.3 42.1

Standard column generation Dual inequalities Procedure RA + dual inequalities

4.00

4.00

3.60

4.00

3.80

3.60

3.60

3.60

3.80

3.80

6.30

3.80

3.70

9.00

% red

tlp

% red

18.3 59.7

2.7 3.6 2.0

33.3 25.9

103.0 83.0 51.4

19.4 50.1

3.2 3.0 2.3

6.3 28.1

Standard column generation Dual inequalities Procedure RA + dual inequalities

94.9 79.0 51.2

16.8 46.0

4.1 3.3 2.6

19.5 36.6

Standard column generation Dual inequalities Procedure RA + dual inequalities

94.3 74.2 41.4

21.3 56.1

2.5 3.3 2.2

32.0 12.0

Standard column generation Dual inequalities Procedure RA + dual inequalities

105.8 86.0 54.2

18.7 48.8

3.0 2.9 2.1

3.3 30.0

Standard column generation Dual inequalities Procedure RA + dual inequalities

91.3 72.3 38.4

20.8 57.9

1.7 1.8 1.1

5.9 35.3

Standard column generation Dual inequalities Procedure RA + dual inequalities

100.2 77.6 43.6

22.6 56.5

1.5 2.3 1.3

53.3 13.3

Standard column generation Dual inequalities Procedure RA + dual inequalities

97.1 75.4 40.6

22.3 58.2

1.9 2.8 1.4

47.4 26.3

Standard column generation Dual inequalities Procedure RA + dual inequalities

91.8 74.6 39.7

18.7 56.8

2.2 2.9 1.8

31.8 18.2

Standard column generation Dual inequalities Procedure RA + dual inequalities

86.7 65.7 17.7

24.2 79.6

1.6 1.4 0.5

12.5 68.8

Standard column generation Dual inequalities Procedure RA + dual inequalities

86.5 65.9 14.3

23.8 83.5

1.4 1.3 0.4

7.1 71.4

Standard column generation Dual inequalities Procedure RA + dual inequalities

88.1 70.7 40.0

19.8 54.6

3.4 3.5 2.1

2.9 38.2

Standard column generation Dual inequalities Procedure RA + dual inequalities

147.0 116.2 64.4

21.0 56.2

7.7 11.2 5.3

45.5 31.2

Standard column generation Dual inequalities Procedure RA + dual inequalities

171.3 129.3 77.9

24.5 54.5

9.2 8.7 8.3

5.4 9.8

Standard column generation Dual inequalities Procedure RA + dual inequalities

79.2 66.0 44.9

16.7 43.3

6.0 5.8 4.6

3.3 23.3

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1349

exists an appreciable number of instances for which the RA procedure gave very good results. In Table 3, we compiled the results obtained with the best 10 instances within each group. This represents 20% of the initial instances. Surprisingly, we were able to cut between 43.3% and 83.5% of the column generation iterations and, even with the disaggregation overhead, the total computing time was reduced by up to 71.4%. In some runs of the column generation algorithm with dual inequalities, we observed an increase of the computing time. This situation was due to a limited number of instances for which the mt1r algorithm found some difficulties. In all the experiments conducted, the n-phase algorithm of Section 4.2 (we will also refer to it as procedure DA) yields far more regular results than those achieved with procedure RA. To illustrate this conclusion, we present the results obtained with a set of well known bin-packing instances for which procedure RA gave only marginal improvements. Three problems sets are considered. The Hard28 problems consists in 28 instances originally generated by Schoenfield [28]. As we can see in Table 4, the standard column generation algorithm takes on average 532.4 iterations to solve them, which is considerable. The two remaining problem sets are the t501 and t249 triplet instances from the already referred OR-library [2]. With the instances of the Hard28 set, we were able to control the number of items in each equivalence group using only the parameters Di. In fact, considering a 4 iterations process, equivalence groups with more than 5 items were rare. This is an a priori guarantee that the time needed to perform the disaggregation will never be high. With the triplet instances, the situation is different. Item sizes differ only in small amounts and, for example, an aggregation with D = 1 leads to a high number of items going to the same equivalence group. Therefore, the maximum number of items per equivalence group was restricted to 5 elements. Since the number of column generation iterations achieved with dual inequalities is reasonably low, we tested a 2 iterations procedure. In Tables 4–6, we present the results obtained with standard column generation, column generation using dual inequalities on the items’ dual variables and procedure DA. In the latter, dual inequalities were also Table 4 Solution data for the Hard28 instances Problem

m

Standard CG

Dual inequalities

splp

tlp

splp

% red

tlp

% red

splp

% red

tlp

% red

6.1 6.9 9.2 5.3 8.4 4.7 7.3 7.1 12.6 10.9 9.3 13.2 7 8.4 5.9 9 5.3 4.8 12.5 10.4 12.4 12.5 12.6 9.3 11.7 14.5 6.7 5.7

290 317 369 207 344 229 363 292 413 446 379 459 220 363 252 329 211 206 371 354 414 374 403 264 357 390 220 208

38.6 36.2 37.9 50.6 38.7 41.9 30.3 43.6 39.4 25.2 31.1 32.4 53.8 33.5 39.6 40.4 46.7 46.2 41.0 35.4 30.8 38.3 36.2 50.0 41.3 42.1 50.3 46.4

4.6 5.4 10.9 3.4 6.2 3.9 6.3 4.8 9.5 13.7 10.5 10.9 3.9 7.1 4.9 6.8 4.1 3.8 9.2 9.2 11.5 10.5 10.5 6.2 9.1 11.2 4.7 4.3

24.6 21.7 18.5 35.8 26.2 17.0 13.7 32.4 24.6 25.7 12.9 17.4 44.3 15.5 16.9 24.4 22.6 20.8 26.4 11.5 7.3 16.0 16.7 33.3 22.2 22.8 29.9 24.6

142 172 199 53 174 87 187 161 260 282 182 251 104 176 142 166 53 86 274 185 273 167 168 154 248 201 55 68

69.9 65.4 66.5 87.4 69.0 77.9 64.1 68.9 61.9 52.7 66.9 63.0 78.2 67.8 65.9 69.9 86.6 77.5 56.4 66.2 54.3 72.4 73.4 70.8 59.2 70.1 87.6 82.5

3.2 5.6 8.2 1.7 4.3 3.5 5.4 4.5 9.9 9.6 6.9 8.9 3.4 5.9 4.2 5.7 2.4 2.9 10.9 7.3 9.9 6.5 6.3 6.6 10.4 7.9 2.4 3.0

47.5 18.8 10.9 67.9 48.8 25.5 26.0 36.6 21.4 11.9 25.8 32.6 51.4 29.8 28.8 36.7 54.7 39.6 12.8 29.8 20.2 48.0 50.0 29.0 11.1 45.5 64.2 47.4

323.0

39.3

7.4

16.9

166.8

68.7

6.0

32.6

bpp14 bpp832 bpp40 bpp360 bpp645 bpp742 bpp766 bpp60 bpp13 bpp195 bpp709 bpp785 bpp47 bpp181 bpp359 bpp485 bpp640 bpp716 bpp119 bpp144 bpp561 bpp781 bpp900 bpp175 bpp178 bpp419 bpp531 bpp814

136 139 144 148 141 148 143 144 161 161 160 163 158 157 164 163 165 158 173 173 177 174 173 185 178 189 175 179

472 497 594 419 561 394 521 518 682 596 550 679 476 546 417 552 396 383 629 548 598 606 632 528 608 673 443 388

Average

161.8

532.4

8.9

Procedure DA + dual inequalities

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1350

added to the aggregated master problems. In the next comments, we will denote these procedures by SCG, CG–DI and DA–DI, respectively.

Table 5 Solution data for the t501 instances Problem

m

Standard CG

Dual inequalities

Procedure DA + dual inequalities

splp

tlp

splp

% red

tlp

% red

splp

% red

tlp

% red

104 87 100 103 96 96 96 91 103 96 92 102 91 104 95 102 95 92 99 95

39.9 42.8 42.5 39.1 42.9 47.3 46.7 45.8 39.4 38.1 44.2 42.7 47.1 44.4 50.0 43.6 46.0 46.2 47.6 47.8

7.5 6.6 8.7 14.1 12.4 7.4 7.4 6.7 8.0 7.2 6.8 7.9 6.5 8.3 7.7 8.1 7.2 7.7 7.4 7.0

33.6 31.3 25.6 10.2 8.8 39.8 40.3 38.0 29.8 26.5 35.8 34.2 40.4 36.6 46.5 37.7 39.5 35.8 40.3 39.7

44 38 43 55 44 54 44 38 47 42 39 44 48 47 45 39 48 43 52 50

74.6 75.0 75.3 67.5 73.8 70.3 75.6 77.4 72.4 72.9 76.4 75.3 72.1 74.9 76.3 78.5 72.7 74.9 72.5 72.5

5.3 4.3 3.6 5.7 6.0 4.5 6.1 4.4 4.4 3.8 4.7 6.1 5.0 6.3 4.8 6.1 6.0 3.7 4.1 5.7

53.1 55.2 69.2 63.7 47.4 63.4 50.8 59.3 61.4 61.2 55.7 49.2 54.1 51.9 66.7 53.1 49.6 69.2 66.9 50.9

44.3

8.0

32.8

45.2

74.0

5.0

58.0

t501_00 t501_01 t501_02 t501_03 t501_04 t501_05 t501_06 t501_07 t501_08 t501_09 t501_10 t501_11 t501_12 t501_13 t501_14 t501_15 t501_16 t501_17 t501_18 t501_19

190 192 190 199 195 195 196 192 196 189 190 195 189 198 203 197 198 196 193 192

173 152 174 169 168 182 180 168 170 155 165 178 172 187 190 181 176 171 189 182

11.3 9.6 11.7 15.7 11.4 12.3 12.4 10.8 11.4 9.8 10.6 12.0 10.9 13.1 14.4 13.0 11.9 12.0 12.4 11.6

Average

194.3

174.1

11.9

97.0

Table 6 Solution data for the t249 instances Problem

m

Standard CG

Dual inequalities

splp

tlp

splp

% red

tlp

% red

splp

% red

tlp

% red

t249_00 t249_01 t249_02 t249_03 t249_04 t249_05 t249_06 t249_07 t249_08 t249_09 t249_10 t249_11 t249_12 t249_13 t249_14 t249_15 t249_16 t249_17 t249_18 t249_19

134 140 139 142 134 145 138 137 139 141 140 141 141 141 145 142 144 145 138 136

146 151 152 150 136 158 148 136 153 155 146 149 146 153 157 145 151 158 144 137

4.2 4.5 4.5 4.5 3.5 5.0 4.1 3.8 4.5 5.0 4.0 4.3 4.2 4.6 5.1 4.3 4.6 4.9 4.1 3.6

82 87 89 75 76 80 84 80 93 92 83 83 83 84 103 80 81 92 82 77

43.8 42.4 41.5 50.0 44.1 49.4 43.2 41.2 39.2 40.7 43.2 44.3 43.2 45.1 34.4 44.8 46.4 41.8 43.1 43.8

2.5 3.8 2.9 2.5 2.3 2.8 2.7 2.6 4.8 3.2 2.7 2.8 2.9 2.8 3.8 2.6 2.8 3.2 2.7 2.6

40.5 15.6 35.6 44.4 34.3 44.0 34.1 31.6 6.7 36.0 32.5 34.9 31.0 39.1 25.5 39.5 39.1 34.7 34.1 27.8

48 49 46 48 42 41 47 42 43 41 49 38 42 42 54 49 43 43 48 41

67.1 67.5 69.7 68.0 69.1 74.1 68.2 69.1 71.9 73.5 66.4 74.5 71.2 72.5 65.6 66.2 71.5 72.8 66.7 70.1

2.1 2.1 2.2 2.3 2.0 3.0 1.6 2.9 2.0 1.8 2.4 2.2 2.0 2.0 2.3 1.9 2.2 2.3 1.6 1.9

50.0 53.3 51.1 48.9 42.9 40.0 61.0 23.7 55.6 64.0 40.0 48.8 52.4 56.5 54.9 55.8 52.2 53.1 61.0 47.2

Average

140.1

148.6

4.4

43.3

3.0

31.8

44.8

69.9

2.1

52.3

84.3

Procedure DA + dual inequalities

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

1351

With procedure DA–DI, the number of column generation iterations needed to reach the optimal solution decreases significantly. Compared to CG–DI, the percentage of reduction is approximately 50% for the three sets of instances. The number of columns in the final restricted master problem increases, but these extra columns are generated faster via the disaggregation processes. Indeed, the computational results show that it is more efficient to perform even a restricted disaggregation rather than generating columns by solving a knapsack problem, whatever the algorithm we choose. For the three sets of instances, the average computing times decrease with procedure DA–DI. For the Hard28 set, standard column generation needs 532.4 iterations on average and runs in 8.9 seconds. With CG–DI, we get the optimal solution 16.9% faster. Procedure DA–DI almost doubles this value with 32.6% of time reduction. For the t501 set, CG–DI yields a reduction of 32.8%, while procedure DA–DI improves this value in 25.2%. The maximum reduction in the computing time is achieved for the instance t501_02 with 69.2%. For this instance, procedure CG–DI yields a reduction of 25.6%. With problems growing in size and complexity, this tendency of time reduction will surely become even more significant. 6. Conclusions We analyzed different strategies to stabilize and accelerate column generation in the context of variable sized bin-packing problems. Some of the prescribed methods are directly applicable to the standard bin-packing problem, the well known special case in which only one bin type is available. New weak and deep dual-optimal inequalities were introduced. We showed that they are a good complement to the dual inequalities imposed on items’ dual variables. Their number is limited and, so, we can normally enumerate them all before the resolution of the first master problem. Additionally, we explored the idea of aggregation to control the progress of the dual variables and thus accelerate the resolution of the master problems. Procedure RA, despite its extreme simplicity, gave very interesting results but was outperformed by the more sophisticated n-phase algorithm of Section 4.2. This latter forces sets of dual variables to be equal during its various stages. The respective approximations are solved by column generation with a reduced tailing off, which leads to an overall process with better convergence properties. Other approaches have been proposed with the purpose of accelerating the column generation algorithm, such as the procedure proposed by Degraeve and Peeters [7], which is based on a simplex method/subgradientoptimization algorithm to solve the LP relaxation of the cutting stock problem. Our approaches can be combined with their algorithm. In fact, we believe that the fastest procedure should be the one that combines them. To evaluate the performance of the approaches, extensive computational experiments were carried out on various sets of randomly generated instances and other instances found in the literature. The appreciable improvements we achieved allow us to claim the effectiveness of these strategies. Acknowledgements We thank two anonymous referees for their constructive comments, which led to a clearer presentation of the material. We also want to thank Gleb Belov and Guntran Scheithauer for providing us the Hard28 set. This work was partially supported by the portuguese Science and Technology Foundation (Projecto POSI/ 1999/ SRI/35568) and by the Algoritmi Research Center of the University of Minho, and was developed in the Industrial and Systems Engineering Group. References [1] C. Alves, J.M. Vale´rio de Carvalho, A stabilized branch-and-price-and-cut algorithm for the multiple length cutting stock problem, Working Paper, Universidade do Minho, Portugal, 2006. [2] J.E. Beasley, OR-library: Distributing test problems by electronic email, Journal of the Operational Research Society 41 (1990) 1069– 1072. [3] G. Belov, G. Scheithauer, A cutting plane algorithm for the one-dimensional cutting stock problem with multiple stock lengths, European Journal of Operational Research 141 (2002) 274–294. ´ cole Polytechnique de Montre´al, Canada, 2002. [4] H. Ben Amor, Stabilisation de l’algorithme de ge´ne´ration de colonnes, PhD Thesis, E

1352

C. Alves, J.M. Vale´rio de Carvalho / European Journal of Operational Research 183 (2007) 1333–1352

[5] H. Ben Amor, J. Desrosiers, J.M. Vale´rio de Carvalho, Dual-optimal inequalities for stabilized column generation, Operations Research, in press. [6] C. Chu, R. La, Variable-sized bin packing: Tight absolute worst-case performance ratios for four approximation algorithms, SIAM Journal on Computing 30 (6) (2001) 2069–2083. [7] Z. Degraeve, M. Peeters, Optimal integer solutions to industrial cutting-stock problems: Part 2, Benchmark results, INFORMS Journal on Computing 15 (1) (2003) 58–81. [8] J. Desrosiers, M. Luebbecke, Selected topics in column generation, Technical Paper, Les Cahiers du GERAD G-2002-64, Canada, 2002. [9] O. du Merle, J. Goffin, J. Vial, On improvements to the analytic center cutting plane method, Computational Optimization and Applications 11 (1998) 37–52. [10] O. du Merle, D. Villeneuve, J. Desrosiers, P. Hansen, Stabilized column generation, Discrete Mathematics 194 (1999) 229–237. [11] H. Dyckhoff, A typology of cutting and packing problems, European Journal of Operational Research 44 (1990) 145–159. [12] D. Friesen, M. Langston, Variable sized bin packing, SIAM Journal on Computing 15 (1986) 222–230. [13] M. Garey, D.S. Johnson, Computers and Intractability. A Guide to the Theory of NP-Completeness, W.H. Freeman and Company, 1979. [14] P.C. Gilmore, R.E. Gomory, A linear programming approach to the cutting stock problem, Operations Research 9 (1961) 849–859. [15] P.C. Gilmore, R.E. Gomory, A linear programming approach to the cutting stock problem – Part ii, Operations Research 11 (1963) 863–888. [16] J.-B. Hiriart-Urruty, C. Lemare´chal, Convex analysis and minimization algorithms II: Advanced theory and bundle methods, Springer-Verlag, Berlin, 1993. [17] O. Holthaus, Decomposition approaches for solving the integer one-dimensional cutting stock problem with different types of standard lengths, European Journal of Operational Research 141 (2002) 295–312. [18] ILOG, CPLEX 6.5, User’s Manual, 1999. [19] B. Kallehauge, J. Larsen, O. Madsen, Lagrangean duality applied on vehicle routing with time windows, Technical Paper, Technical University of Denmark, Informatics and Mathematical Modelling IMM-TR-2001-9, Kgs Lyngby, Denmark, 2001. [20] L. Kantorovich, Mathematical methods of organising and planning production (translated from a report in Russian, dated 1939), Management Science 6 (1960) 366–422. [21] R. Marsten, W. Hogan, J. Blankenship, The Boxstep method for large-scale optimization, Operations Research 23 (1975) 389–405. [22] S. Martello, P. Toth, Knapsack Problems: Algorithms and Computer Implementations, Wiley, New York, 1990. [23] M. Monaci. Algorithms for packing and scheduling problems, PhD Thesis, Universita` di Bologna, 2002. [24] F. Murgolo, An efficient approximation scheme for variable-sized bin packing, SIAM Journal on Computing 16 (1987) 149–161. [25] G.L. Nemhauser, L.A. Wolsey, Integer and Combinatorial Optimization, Wiley, New York, 1988. [26] G. Roodman, Near optimal solutions to one-dimensional cutting stock problem, Computers and Operations Research 13 (1986) 713– 719. [27] D. Rogers, R. Plante, R. Wong, J. Evans, Aggregation and disaggregation techniques and methodology in optimization, Operations Research 39 (1991) 553–582. [28] J.E. Schoenfield, Fast, exact solution of bin-packing problems without linear programming, Technical Paper, US Army Space and Missile Defense Command, Alabama, USA, 2002. [29] J.M. Vale´rio de Carvalho, Exact solution of bin-packing problems using column generation and branch-and-bound, Annals of Operations Research 86 (1999) 629–659. [30] J.M. Vale´rio de Carvalho, LP models for bin-packing and cutting stock problems, European Journal of Operational Research 141 (2002) 253–273. [31] J.M. Vale´rio de Carvalho, Using extra dual cuts to accelerate convergence in column generation, INFORMS Journal on Computing 17 (2) (2005) 175–182. [32] P. Vance, Branch-and-price algorithms for the one-dimensional cutting stock problem, Computational Optimization and Applications 9 (1998) 211–228. [33] D. Villeneuve, J. Desrosiers, M. Lu¨bbecke, F. Soumis, On compact formulations for integer programs solved by column generation, Technical Paper, Les Cahiers du GERAD G-2003-6, Canada, 2003. [34] G. Wa¨scher, H. Haussner, H. Schumann, An improved typology of cutting and packing problems, European Journal of Operational Research, this issue, doi:10.1016/j.ejor.2005.12.047.