- Email: [email protected]
Solving combinatorial optimization problems with line-up competition algorithm
Computers and Chemical Engineering 27 (2003) 251 /258 www.elsevier.com/locate/compchemeng
Solving combinatorial optimization problems with line-up competition algorithm Liexiang Yan Department of Chemical Engineering, Wuhan University of Technology, Wuhan 430070, People’s Republic of China Received 26 March 2001; received in revised form 6 December 2001; accepted 3 September 2002
Abstract The line-up competition algorithm (LCA), a simulated evolutionary algorithm, is proposed for solving combinatorial optimization problems. In the LCA, there exist independent and parallel families during evolution. Two means of competitions, the competitions between families and inside a family, are used to increase the search efficiency. Again, all families are ranked a lineup based on their objective function, and allocated different search spaces according to their position in the line-up, which is favorable to balance local and global search. The LCA algorithm is applied to the solution of knapsack problem and the optimal design of pressure relief header network. Computational results reveal that the LCA algorithm is able to find optimal or nearoptimal solutions after examining an extremely small fraction of search space. # 2002 Elsevier Science Ltd. All rights reserved. Keywords: Line-up competition algorithm; Evolutionary algorithm; Combinatorial optimization; Knapsack problem; Pipe network
1. Introduction Combinatorial optimization is the optimization of functions with discrete variables. A large number of practical problems come from different areas such as management science, operations research, computer science, and engineering. Some of these problems belong to a class of NP-complete problems. Because of the complexity of these problems, many techniques for finding sub-optimal or near-optimal solutions have been proposed (Papadimitriou & Steiglitz, 1982). Recently some other optimization techniques, such as simulated annealing (SA) and simulated evolution, have also been proposed for solving combinatorial optimization problems (Kirkpatrick, Gelatt & Vecchi, 1983; Goldberg, 1989; Cardoso, Salcedo & de Feyo, 1994; Yip & Pao, 1995). In the chemical process system engineering area, SA and simulated evolution algorithms have been applied to the synthesis and design (Dolan, Cumings & Levan, 1989; Androulakis & Venkatasubramenian, 1991; Cardoso et al., 1994; Floquet, Pibouleau & Doemenech, 1994; Hanagandi & E-mail address: [email protected] (L. Yan).
Nikolaou, 1998), to the development of process models (Greeff & Aldrich, 1998; McKay, Willis & Barton, 1997), and to the design and scheduling of multi-product batch chemical process (Das, Cummings & Levan, 1990; Patel, Mah & Karlmi, 1991; Ku & Karrimi, 1991; Wang, Quan & Xu, 1995; Jung, Lee & Lee, 1998). In this paper, the line-up competition algorithm (LCA; Yan, 1998; Yan & Ma, 2001), a simulated evolutionary algorithm, is proposed for solving combinatorial optimization problem. First, the framework of solving combinatorial problems with the LCA is given. Then, the knapsack problem and the pressure relief header network problem, two typical combinatorial optimization problems, are used for illustrating the effectiveness of the LCA algorithm. The superior performance of the LCA algorithm is analyzed in Section 5.
2. LCA algorithm The LCA algorithm is an evolutionary algorithm (Yan, 1998; Yan & Ma, 2001). However, it is different from the other evolutionary algorithms in nature. In the LCA algorithm, independent and parallel evolutionary
0098-1354/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved. PII: S 0 0 9 8 - 1 3 5 4 ( 0 2 ) 0 0 1 9 4 - 1
252
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
Nomenclature bi d C F f g L Le M PA PB R Si T W X Greek symbols a b m
pressure constraint of node i (Pa2) pipe segment inside diameter (m) capability for knapsack problem objective function friction factor penalty function physical length of pipe segment (m) equivalent length of segment (m) molecular weight (kg/kg mol) upstream pressure of pipe segment (Pa) downstream pressure of pipe segment (Pa) universal gas constant (J/kg mol K) size of the ith object for knapsack problem absolute temperature of stream formed by mixing (K) mass flow rate (kg/s) n -dimension vector with element 0 or 1 cost coefficient ($/m) cost coefficient ($/m2) fluid viscosity (kg/m s)
families are always kept during evolution, each family producing offspring only by asexual reproduction. Moreover, there are the two levels of competition in the algorithm. One is the survival competition inside a family. The best one of each family survives in each generation. The other level is the competition between families. According to their values of objective function, families are ranked a line-up. The best family is in the first position in the line-up, while the worst family is in the final position. The families of different position have different driving forces of competition. The driving force of competition may be understood as the power of impelling family mutation. For different optimization problems, it has different expressions. By the above two levels of the competitions, the first family in the line-up is replaced continually by other families, or the value of its objective function is updated continually. As a result, the optimal solution can be approached rapidly. For the solution of combinatorial optimization problem, the LCA algorithm can be described as follows: Step 1. Randomly generate m families (so-called parents). Step 2. Compute the value of objective function for each family. Step 3. According to the values of objective function, the m families are ranked to form a line-up. For finding the problems of global minimum, the line-up is arranged as an ascending sequence. Otherwise, for
finding the problems of global maximum, the line-up as a descending sequences. Step 4. Based on the position of each family in the line-up, decide the times of its mutations. The first family in the line-up has the fewest times of mutations, while the final family has the most times of mutations. Through this step operation, get m children generated from m parents. Step 5. Compare the values of objective function of children and parent for each family, and better one survives as parent in the next generation. Step 6. Repeat steps 3/5 until specified evolutionary generations. Step 4 is the core of the algorithm. Its action is to balance local and global search. The reason is that the first family in the line-up mutates only a time, which is beneficial to speed up local search for finding optimal solution quickly, and that final family mutates many times, resulting in migration in a larger search space, which pays an important role in global search. The flowchart of the LCA algorithm is shown in Fig. 1. Here, The given algorithm is only a framework. Different mutation neighborhoods may be constructed according to different combinatorial optimal problems. It is very important to construct a rational mutation neighborhood in practical use. Based on the idea of the LCA algorithm, the two principles should be followed when constructing a mutation neighborhood.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
253
fill the knapsack as many as possible. Mathematical form is given by: max
n X
Si Xi
i 1
subject to the constraints: n X
Si Xi 5C
Xi f0; 1g 1 5i 5n
(1)
i 1
By using penalty function method to handle the above constraint, knapsack problem is converted into solving the following problem: maxF (X )
n X
Si Xi g(X )
(2)
i 1
where g (X ) is called as penalty function, it has the following form: 8 n X > > >0; C Si Xi ]0 > < i 1 (3) g(X ) n n > X X > > > C Si Xi ; Si Xi 5 0 :a C i 1
Fig. 1. Flowchart of LCA (xij (k ) represents the i th family in the j th position of line-up; N is the number of generations).
where a is penalty coefficient.
1) When a family mutates a time or migrates at a small step, it will not produce distinct change to system configuration. 2) When a family mutates many times or migrates at a stride, it will lead to distinct change to system configuration. Namely, the change extent of system configuration is proportional to the number of mutations.
3.2. Mutation neighborhood
In the LCA, The number of families, m , is a very important parameter. It influences directly on the balance of local and global search. A smaller m is to benefit local search, but to go against global search. A larger one can provide better global search, but handicaps local search. We have to trade between the local and global search. Based on our computing experiences, it is appropriate for m to be controlled between 5 and 10% of the number of integer variables related problem.
3. Solving knapsack problem
i 1
According to the above two principles of constructing mutation neighborhood, we range Si from large to small in order to bring less change to system configuration when a family mutates a time. S1 ]S2 ] . . . Sn Above sequence corresponds to n-dimensional vector X /[X1, X2, . . ., Xn ], i.e. element Xi corresponds to Si . When Xi /1, it denotes that Si is placed in the knapsack. When Xi /0, it denotes that Si is not chosen. On the above basis, we propose the following mutation neighborhood. Randomly select a certain element Xi in the vector X ; if Xi /0, let Xi /1 and Xi 1 /0; if Xi /1, let Xi /0 and Xi1 /1. Expression is given by: Xi 1 Before mutation * After mutation 0 where symbol ‘*’ denotes uncertain
Xi 0 1 value
Xi 1 0 0 or
Xi1 * 1 1.
3.1. Problem description 3.1.1. 0/1 Knapsack problem Given several objects with the sizes S1, S2, . . ., Sn and a knapsack with capability C , where S1, S2, . . ., Sn and C are positive integers. A subset of n objects is found to
3.3. Case study Solve the below knapsack problem of 100 objects with capacity C /6666.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
254
8 9 253; 245; 243; 239; 239; 239; 238; 238; 237; > > > > > > >232; 231; 231; 230; 229; 228; 227; 224; 217; > > > > > > > > > >213; 207; 203; 201; 195; 194; 191; 187; > > > > > > > 187; 177; 175; 171; 169; 168; 166; 164; 161; 160; > > > > < = 158; 150; 149; 147; 141; 140; 139; 136; 135; S >132; 128; 126; 122; 120; 119; 116; 116; 114; > > > > > > > 111; 110; 105; 105; 104; 103; 93; 92; 90; 79; > > > > > > > > > > 78; 77; 76; 76; 75; 73; 62; 62; 61; 60; 60; 59; > > > > > > > > 57; 56; 53; 53; 51; 50; 44; 44; 42; 42; 38; 36; > > : ; 34; 28; 27; 24; 22; 18; 12; 10; 7; 4; 4; 1
In order to solve the above problem, m vectors X s were produced randomly, which represent m families. The element values in each vector X were set to 0 or 1 at the probability of 50%. Table 1 gives the statistical results obtained with 1, 3, 5, 10 and 15 families, respectively, for 100 runs. Here, the same number of function evaluations was used for each different m . The search process of one family is purely a local search, which the successes of finding the optimal solution are 92. The successes are 100, 100, 97 and 93, respectively, for 3, 5, 10 and 15 families. It is clear that multi-families have better robustness than one family. However, too large families will result in the decrease in robustness. From Table 1, note also that the computational time of multi-families is much less than that of one family. Besides, with the increase of the number of families, the computational time decreases first, and reaches the least value when m /10. Then it increases with the increase of the number of families. Generally, the search efficiency of multi-families is superior to that of one family. But the value of m should be controlled inside an appropriate region because too large families will result in the decrease in robustness and the increase in computational time. For this problem, five families are obviously a better choice. Fig. 2 shows the processes of evolution for five families. Initially, the values of objective function were {5291, 5522, 5466, 6470, 5885}. Through the evolution of 30 generations, they became {6666, 6666, 6665, 6661.4, 6650.4}. Of the five families, two families reached optimal solutions. One of the optimal solutions is given as follows:
Fig. 2. The courses of evolution for five families for solving knapsack problem.
8 0; > > > > 0; > > < 0; X >1; > > > 1; > > : 0; n X
1; 1; 1; 0; 0; 0;
0; 1; 0; 1; 0; 1;
9 0; 1; 0; 1; 0; 1; 0; 1; 1; 0; 1; 1; 0; 1; > > > 0; 0; 1; 1; 0; 1; 1; 1; 0; 0; 1; 1; 0; 0;> > > = 0; 0; 1; 0; 1; 1; 1; 1; 1; 1; 0; 1; 0; 1; 0; 1; 1; 0; 0; 1; 1; 1; 1; 0; 1; 0; 0; 0;> > > 1; 1; 0; 1; 0; 0; 1; 0; 1; 1; 0; 1; 0; 1;> > > ; 0; 0; 0; 1; 1; 0; 0; 1; 1; 1; 0; 1; 1; 1
Si Xi 6666
i 1
In order to illustrate the role of competition mechanism in the LCA algorithm, we compared the competition with the non-competition. In the latter m families were in the same position and each family mutated only a time. In this case, the LCA algorithm was similar to evolutionary strategy denoted by the term (1, 1)-ES (Nissen, 1994). The number of optimal solutions found was only 26 in 100 runs without the competition between families. Fig. 3 shows the processes of evolution for five families. Through the evolution of 30 generations, the initial values of objective function {6640, 6511, 6466, 6143, 6323} changed into {6665.4, 6661.4, 6661, 651.8, 6638}, none of which evolves to optimal solution.
Table 1 Statistical results obtained with different families (penalty coefficient a 1) m
N
Nf
CPU (s)
Successes
1 3 5 10 15
150 50 30 15 10
150 150 150 150 150
0.38 0.27 0.24 0.22 0.23
92 100 100 97 93
m , Number of families; N , number of generations; Nf, number of function evaluations (Nf m N ); CPU, computational time on 586 computer (150 MHz).
Fig. 3. The courses of evolution for five families in the case of noncompetition for solving knapsack problem.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
From two different processes of evolution in Figs. 2 and 3, it can be seen that the competition mechanism in the LCA algorithm plays a very important role. The introduction of competition mechanism improves largely the solution quality and advances the search efficiency.
4. Optimization of pressure relief header network
pipe segments and 34 fluid source nodes. The decision variables are the pipe diameters, which may take one of nine available discrete values for each segment. The number of possible unconstrained designs is 979, which is indeed a very large number. It is very difficult to use enumerate method for finding optimal solution because it is too time-consuming. The pressures in the upstream (A) and downstream (B) end of a pipe segment are approximated by: p2A p2B
4.1. Problem description In this section, we examine the problem of optimizing a pressure relief header network, which is used for a safety system in refineries and chemical plants. The role of the network is to transport fluids as expediently as possible from process equipment to the flare. In order to achieve sonic transport velocity through a relief valve for some transitory period, the pressure on downstream side of the valve must be remained below a certain value. Therefore, there is a maximum allowable pressure drop of each valve-flare path along the network, which constitutes a set of constraints for optimization of the problem. Fig. 4 is a pressure relief header network. Cheng and Mah (1976); Dolan et al. (1989) solved this problem with discrete merging method. Cardoso et al. (1994) applied SA to the problem. The network has 79 fixed-length
255
0:3697f (L Le )W 2 RT p2 Md 5
(4)
where L is the physical length of the pipe section; Le, the equivalent length; W , the mass flow rate; T , the temperature; M , the molecular weight; d , the pipe diameter; R , the gas constant; and f is a non-friction factor given by: f 0:0475
1:1744 10 12 W md
0:186
(5)
where m is fluid viscosity. The temperature and molecular weights corresponding to the various pipe sections are determined by a simple mixing rule (Dolan et al., 1989). The network cost is determined by the capital cost of the pipe segments. The objective is to minimize the total cost. The objective function is given by:
Fig. 4. Pressure relief header network.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
256
min F
79 X
(abdi )Li
(6)
i 1
where a and b are constants, subject to the pressure constraints: X bi ] (p2A p2B ) i 1; 2; . . . ; 34 (7) where bi is the maximum allowable pressure drop between node i and the flare. 4.2. Problem solution To solve the problem with the LCA algorithm, we adopted the approach of Dolan et al. (1989) to construct mutation neighborhood, i.e. a pipe segment is selected randomly, and then its diameter is increased or decreased by 1 unit at the probability of 50%, respectively. In this way, the mutation of a time will not produce distinct change in the network cost, but the mutation of a few times leads possibly to notable change. Therefore, the above mutation neighborhood is to follow the basic principles of the LCA algorithm. The pressure constraints were handled by using the approach of penalty function. We used m families for representing m design configurations. In the progress of evolution, each family reproduced only one offspring in each generation. Initially each family had the same design configuration as that given in the literature (Dolan et al., 1989), where all pipe diameters were set to the largest possible value. Table 2 shows the statistical results obtained with 1, 5, 10 and 20 families, respectively, for 30 runs. Here, the same number of function evaluations was used for each different m for the purpose of comparison. Note that five families obtain the lowest cost and have the best robustness, with only a somewhat increases in computational time compared with 10 and 20 families. One family, corresponding to local search, is the worst and requires much more computational time than the multi-families. By comparison, we can state that the competition of multi-families increases significantly the solution quality and search
Fig. 5. The course of evolution for five families for optimizing pressure relief header network.
Table 3 Comparison of the LCA with DM and SA Algorithms a
DM SAb LCA a b
Nf
Average cost ($)
Best cost ($)
S.D.
/ 5 024 163 20 000
/ 165 353 164 210
165 018 165 018 163 970
/ 177 132
DM (Dolan et al., 1989). SA (Cardoso et al., 1994).
efficiency. Fig. 5 shows the course of evolution for five families. Table 3 gives the comparison of the LCA with discrete merge (DM; Dolan et al., 1989) and SA (Cardoso et al., 1994). The results show that the network cost of $163 970 obtained with the LCA is less than that of $165 018 with DM and SA, respectively. In addition, the LCA requires much fewer the number of function evaluations than SA, being no more than 0.4% of SA. Table 4 gives the best solution obtained, together with the solution of DM and SA.
Table 2 Statistical results obtained with different m families (a 7.1243 $/m; b 341.05 $/m2) m
N
Nf
CPU (s)
Average cost
Best cost ($)
S.D.
1 5 10 20
20 000 4000 2000 1000
20 000 20 000 20 000 20 000
405.6 263.6 247.6 249.4
166 540 164 210 164 250 164 632
165 230 163 970 164 010 164 280
1128 132 185 226
m , Number of families; N , number of generations; Nf, number of function evaluations (Nf m N ); CPU, computational time on 586 computer (150 MHz).
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
257
Table 4 Optimal solutions of DM, SA and LCA Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA)
1 1 1 1 13 2 2 2 25 1 1 1 37 2 2 2 49 2 2 2 61 1 1 1 73 1 3 1
2 2 2 2 14 2 2 2 26 1 1 1 38 3 3 2 50 3 3 3 62 4 4 3 74 3 3 3
3 2 2 2 15 2 2 2 27 1 1 1 39 2 2 2 51 2 2 2 63 2 2 2 75 1 1 1
4 1 1 1 16 2 2 2 28 1 1 1 40 3 3 3 52 3 3 3 64 2 2 2 76 4 4 5
5. Conclusions The framework of solving a combinatorial optimization problem with the LCA algorithm is proposed in this paper. The two basic principles determined mutation neighborhoods are given. The applications of the proposed algorithm to the knapsack problem and the optimization of pressure relief header network problem reveals that the LCA algorithm is able to find optimal or near-optimal solution after examining an extremely small fraction of search space. The main performance of the LCA algorithm is able to balance well local and global search. By ordering, in line-up the front families, which have obtained better solutions, are allocated fewer mutation times, which is favorable to speed up local search and accelerate finding optimal solution. The hind families, which have obtained worse solutions, are allocated more mutation times, which is helpful for global search. In particular, the LCA algorithm has the capability of overcoming local minimum. Even if all families gather round a local minimum, it can also surpass the local minimum. This is because the families of different positions in a line-up have the different mutation times, i.e. different search spaces. Owing to having larger search space, the hind families in the line-up likely find better solutions than this local minimum. Once the hind
5 2 2 2 17 1 2 1 29 2 2 2 41 1 1 1 53 2 2 2 65 2 2 2 77 2 2 2
6 1 1 1 18 1 2 1 30 1 2 1 42 3 3 3 54 2 2 2 66 3 3 2 78 3 3 3
7 1 1 1 19 1 1 1 31 2 2 2 43 4 4 4 55 2 2 2 67 2 2 1 79 7 7 7
8 1 1 1 20 1 1 1 32 1 1 1 44 4 6 4 56 3 3 2 68 3 3 3 / / / /
9 1 1 1 21 1 1 1 33 1 1 1 45 1 1 1 57 2 2 2 69 1 1 1 / / / /
10 1 1 1 22 1 3 1 34 1 1 1 46 2 1 1 58 4 4 3 70 4 3 3 / / / /
11 1 1 1 23 1 1 1 35 2 2 2 47 1 1 1 59 1 1 1 71 1 1 1 / / / /
12 1 1 1 24 1 1 1 36 2 2 2 48 2 2 2 60 4 4 3 72 3 3 3 / / / /
families escape the local minimum, the positions of these families will be moved to the front of the line-up. As a result, the original families in the front of the line-up will be moved to rear, which lets them obtain more mutation times, thus, they can also escape the local minimum. This characteristic embodies the unification of competition and cooperation between families. Because of the above characteristic, all families chase each other, and their positions in the line-up ascend alternately. Through competitions, the first family in the line-up is replaced continually by other families, or the value of objective function of the first position is updated continually. As a result, the optimal solution is approached rapidly.
References Androulakis, I. P., & Venkatasubramenian, V. (1991). A genetic algorithmic framework for process design and optimization. Computers and Chemical Engineering 15 , 217 /228. Cardoso, M. F., Salcedo, R. L., & de Feyo, A. (1994). Nonequilibrium simulated annealing: a faster approach to combinatorial minimization. Industrial Engineering and Chemical Research 33 , 1908 /1918. Cheng, W. B., & Mah, R. S. H. (1976). Optimal design of pressure relieving piping networks by discrete merging. American Institute of Chemical Engineering Journal 22 (3), 471 /476.
258
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
Das, H., Cummings, P. T., & Levan, M. D. (1990). Scheduling of serial multiproduct batch processes via simulated annealing. Computers and Chemical Engineering 14 , 1351 /1362. Dolan, W. B., Cumings, P. T., & Levan, M. D. (1989). Process optimization via simulated annealing: application to network design. American Institute of Chemical Engineering Journal 35 , 725 /736. Floquet, P., Pibouleau, L., & Doemenech, S. (1994). Separation sequence synthesis: how to use simulated annealing procedure. Computers and Chemical Engineering 18 , 1141 /1148. Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning . Addison-Wesley. Greeff, D. J., & Aldrich, C. (1998). Empirical modeling of chemical process system with evolutionary programming. Computers and Chemical Engineering 22 , 995 /1005. Hanagandi, V., & Nikolaou, M. (1998). A hybrid approach to global optimization using a clustering algorithm in a genetic search framework. Computers and Chemical Engineering 22 , 1913 / 1925. Jung, J. H., Lee, C. H., & Lee, I. B. (1998). A genetic algorithm for scheduling of multi-product batch processes. Computers and Chemical Engineering 22 , 1725 /1730. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulation annealing. Science 220 , 671 /680.
Ku, H. M., & Karrimi, I. A. (1991). An evaluation of simulated annealing for batch process scheduling. Industrial Engineering and Chemical Research 30 , 163 /169. McKay, B., Willis, M., & Barton, G. (1997). Steady-state modeling of chemical process systems using genetic programming processes. Computers and Chemical Engineering 21 , 981 /996. Nissen, V. (1994). Solving the quadratic assignment problem with clues from nature. IEEE Transactions on Neural Network 5 , 66 /72. Papadimitriou, C. H., & Steiglitz, K. (1982). Combinatorial optimization algorithms and complexity . Printice-Hall. Patel, A. N., Mah, R. S. K., & Karlmi, I. A. (1991). Preliminary design of multiproduct non-continuous plants using simulated annealing. Computers and Chemical Engineering 15 , 451 /469. Wang, C., Quan, H., & Xu, X. (1995). Optimal design of multiproduct batch chemical process using genetic algorithms. Industrial Engineering and Chemical Research 35 , 3560 /3566. Yan, L. X. (1998). Ph.D. dissertation , Beijing: University of Chemical Technology. Yan, L. X., & Ma, D. X. (2001). Global optimization of nonconvex nonlinear programs using line-up competition algorithm. Computers and Chemical Engineering 25 , 1601 /1610. Yip, P., & Pao, Y. H. (1995). Combinatorial optimization with use of guided evolutionary simulated annealing. IEEE Transactions on Neural Networks 6 , 290 /295.
Solving combinatorial optimization problems with line-up competition algorithm Liexiang Yan Department of Chemical Engineering, Wuhan University of Technology, Wuhan 430070, People’s Republic of China Received 26 March 2001; received in revised form 6 December 2001; accepted 3 September 2002
Abstract The line-up competition algorithm (LCA), a simulated evolutionary algorithm, is proposed for solving combinatorial optimization problems. In the LCA, there exist independent and parallel families during evolution. Two means of competitions, the competitions between families and inside a family, are used to increase the search efficiency. Again, all families are ranked a lineup based on their objective function, and allocated different search spaces according to their position in the line-up, which is favorable to balance local and global search. The LCA algorithm is applied to the solution of knapsack problem and the optimal design of pressure relief header network. Computational results reveal that the LCA algorithm is able to find optimal or nearoptimal solutions after examining an extremely small fraction of search space. # 2002 Elsevier Science Ltd. All rights reserved. Keywords: Line-up competition algorithm; Evolutionary algorithm; Combinatorial optimization; Knapsack problem; Pipe network
1. Introduction Combinatorial optimization is the optimization of functions with discrete variables. A large number of practical problems come from different areas such as management science, operations research, computer science, and engineering. Some of these problems belong to a class of NP-complete problems. Because of the complexity of these problems, many techniques for finding sub-optimal or near-optimal solutions have been proposed (Papadimitriou & Steiglitz, 1982). Recently some other optimization techniques, such as simulated annealing (SA) and simulated evolution, have also been proposed for solving combinatorial optimization problems (Kirkpatrick, Gelatt & Vecchi, 1983; Goldberg, 1989; Cardoso, Salcedo & de Feyo, 1994; Yip & Pao, 1995). In the chemical process system engineering area, SA and simulated evolution algorithms have been applied to the synthesis and design (Dolan, Cumings & Levan, 1989; Androulakis & Venkatasubramenian, 1991; Cardoso et al., 1994; Floquet, Pibouleau & Doemenech, 1994; Hanagandi & E-mail address: [email protected] (L. Yan).
Nikolaou, 1998), to the development of process models (Greeff & Aldrich, 1998; McKay, Willis & Barton, 1997), and to the design and scheduling of multi-product batch chemical process (Das, Cummings & Levan, 1990; Patel, Mah & Karlmi, 1991; Ku & Karrimi, 1991; Wang, Quan & Xu, 1995; Jung, Lee & Lee, 1998). In this paper, the line-up competition algorithm (LCA; Yan, 1998; Yan & Ma, 2001), a simulated evolutionary algorithm, is proposed for solving combinatorial optimization problem. First, the framework of solving combinatorial problems with the LCA is given. Then, the knapsack problem and the pressure relief header network problem, two typical combinatorial optimization problems, are used for illustrating the effectiveness of the LCA algorithm. The superior performance of the LCA algorithm is analyzed in Section 5.
2. LCA algorithm The LCA algorithm is an evolutionary algorithm (Yan, 1998; Yan & Ma, 2001). However, it is different from the other evolutionary algorithms in nature. In the LCA algorithm, independent and parallel evolutionary
0098-1354/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved. PII: S 0 0 9 8 - 1 3 5 4 ( 0 2 ) 0 0 1 9 4 - 1
252
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
Nomenclature bi d C F f g L Le M PA PB R Si T W X Greek symbols a b m
pressure constraint of node i (Pa2) pipe segment inside diameter (m) capability for knapsack problem objective function friction factor penalty function physical length of pipe segment (m) equivalent length of segment (m) molecular weight (kg/kg mol) upstream pressure of pipe segment (Pa) downstream pressure of pipe segment (Pa) universal gas constant (J/kg mol K) size of the ith object for knapsack problem absolute temperature of stream formed by mixing (K) mass flow rate (kg/s) n -dimension vector with element 0 or 1 cost coefficient ($/m) cost coefficient ($/m2) fluid viscosity (kg/m s)
families are always kept during evolution, each family producing offspring only by asexual reproduction. Moreover, there are the two levels of competition in the algorithm. One is the survival competition inside a family. The best one of each family survives in each generation. The other level is the competition between families. According to their values of objective function, families are ranked a line-up. The best family is in the first position in the line-up, while the worst family is in the final position. The families of different position have different driving forces of competition. The driving force of competition may be understood as the power of impelling family mutation. For different optimization problems, it has different expressions. By the above two levels of the competitions, the first family in the line-up is replaced continually by other families, or the value of its objective function is updated continually. As a result, the optimal solution can be approached rapidly. For the solution of combinatorial optimization problem, the LCA algorithm can be described as follows: Step 1. Randomly generate m families (so-called parents). Step 2. Compute the value of objective function for each family. Step 3. According to the values of objective function, the m families are ranked to form a line-up. For finding the problems of global minimum, the line-up is arranged as an ascending sequence. Otherwise, for
finding the problems of global maximum, the line-up as a descending sequences. Step 4. Based on the position of each family in the line-up, decide the times of its mutations. The first family in the line-up has the fewest times of mutations, while the final family has the most times of mutations. Through this step operation, get m children generated from m parents. Step 5. Compare the values of objective function of children and parent for each family, and better one survives as parent in the next generation. Step 6. Repeat steps 3/5 until specified evolutionary generations. Step 4 is the core of the algorithm. Its action is to balance local and global search. The reason is that the first family in the line-up mutates only a time, which is beneficial to speed up local search for finding optimal solution quickly, and that final family mutates many times, resulting in migration in a larger search space, which pays an important role in global search. The flowchart of the LCA algorithm is shown in Fig. 1. Here, The given algorithm is only a framework. Different mutation neighborhoods may be constructed according to different combinatorial optimal problems. It is very important to construct a rational mutation neighborhood in practical use. Based on the idea of the LCA algorithm, the two principles should be followed when constructing a mutation neighborhood.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
253
fill the knapsack as many as possible. Mathematical form is given by: max
n X
Si Xi
i 1
subject to the constraints: n X
Si Xi 5C
Xi f0; 1g 1 5i 5n
(1)
i 1
By using penalty function method to handle the above constraint, knapsack problem is converted into solving the following problem: maxF (X )
n X
Si Xi g(X )
(2)
i 1
where g (X ) is called as penalty function, it has the following form: 8 n X > > >0; C Si Xi ]0 > < i 1 (3) g(X ) n n > X X > > > C Si Xi ; Si Xi 5 0 :a C i 1
Fig. 1. Flowchart of LCA (xij (k ) represents the i th family in the j th position of line-up; N is the number of generations).
where a is penalty coefficient.
1) When a family mutates a time or migrates at a small step, it will not produce distinct change to system configuration. 2) When a family mutates many times or migrates at a stride, it will lead to distinct change to system configuration. Namely, the change extent of system configuration is proportional to the number of mutations.
3.2. Mutation neighborhood
In the LCA, The number of families, m , is a very important parameter. It influences directly on the balance of local and global search. A smaller m is to benefit local search, but to go against global search. A larger one can provide better global search, but handicaps local search. We have to trade between the local and global search. Based on our computing experiences, it is appropriate for m to be controlled between 5 and 10% of the number of integer variables related problem.
3. Solving knapsack problem
i 1
According to the above two principles of constructing mutation neighborhood, we range Si from large to small in order to bring less change to system configuration when a family mutates a time. S1 ]S2 ] . . . Sn Above sequence corresponds to n-dimensional vector X /[X1, X2, . . ., Xn ], i.e. element Xi corresponds to Si . When Xi /1, it denotes that Si is placed in the knapsack. When Xi /0, it denotes that Si is not chosen. On the above basis, we propose the following mutation neighborhood. Randomly select a certain element Xi in the vector X ; if Xi /0, let Xi /1 and Xi 1 /0; if Xi /1, let Xi /0 and Xi1 /1. Expression is given by: Xi 1 Before mutation * After mutation 0 where symbol ‘*’ denotes uncertain
Xi 0 1 value
Xi 1 0 0 or
Xi1 * 1 1.
3.1. Problem description 3.1.1. 0/1 Knapsack problem Given several objects with the sizes S1, S2, . . ., Sn and a knapsack with capability C , where S1, S2, . . ., Sn and C are positive integers. A subset of n objects is found to
3.3. Case study Solve the below knapsack problem of 100 objects with capacity C /6666.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
254
8 9 253; 245; 243; 239; 239; 239; 238; 238; 237; > > > > > > >232; 231; 231; 230; 229; 228; 227; 224; 217; > > > > > > > > > >213; 207; 203; 201; 195; 194; 191; 187; > > > > > > > 187; 177; 175; 171; 169; 168; 166; 164; 161; 160; > > > > < = 158; 150; 149; 147; 141; 140; 139; 136; 135; S >132; 128; 126; 122; 120; 119; 116; 116; 114; > > > > > > > 111; 110; 105; 105; 104; 103; 93; 92; 90; 79; > > > > > > > > > > 78; 77; 76; 76; 75; 73; 62; 62; 61; 60; 60; 59; > > > > > > > > 57; 56; 53; 53; 51; 50; 44; 44; 42; 42; 38; 36; > > : ; 34; 28; 27; 24; 22; 18; 12; 10; 7; 4; 4; 1
In order to solve the above problem, m vectors X s were produced randomly, which represent m families. The element values in each vector X were set to 0 or 1 at the probability of 50%. Table 1 gives the statistical results obtained with 1, 3, 5, 10 and 15 families, respectively, for 100 runs. Here, the same number of function evaluations was used for each different m . The search process of one family is purely a local search, which the successes of finding the optimal solution are 92. The successes are 100, 100, 97 and 93, respectively, for 3, 5, 10 and 15 families. It is clear that multi-families have better robustness than one family. However, too large families will result in the decrease in robustness. From Table 1, note also that the computational time of multi-families is much less than that of one family. Besides, with the increase of the number of families, the computational time decreases first, and reaches the least value when m /10. Then it increases with the increase of the number of families. Generally, the search efficiency of multi-families is superior to that of one family. But the value of m should be controlled inside an appropriate region because too large families will result in the decrease in robustness and the increase in computational time. For this problem, five families are obviously a better choice. Fig. 2 shows the processes of evolution for five families. Initially, the values of objective function were {5291, 5522, 5466, 6470, 5885}. Through the evolution of 30 generations, they became {6666, 6666, 6665, 6661.4, 6650.4}. Of the five families, two families reached optimal solutions. One of the optimal solutions is given as follows:
Fig. 2. The courses of evolution for five families for solving knapsack problem.
8 0; > > > > 0; > > < 0; X >1; > > > 1; > > : 0; n X
1; 1; 1; 0; 0; 0;
0; 1; 0; 1; 0; 1;
9 0; 1; 0; 1; 0; 1; 0; 1; 1; 0; 1; 1; 0; 1; > > > 0; 0; 1; 1; 0; 1; 1; 1; 0; 0; 1; 1; 0; 0;> > > = 0; 0; 1; 0; 1; 1; 1; 1; 1; 1; 0; 1; 0; 1; 0; 1; 1; 0; 0; 1; 1; 1; 1; 0; 1; 0; 0; 0;> > > 1; 1; 0; 1; 0; 0; 1; 0; 1; 1; 0; 1; 0; 1;> > > ; 0; 0; 0; 1; 1; 0; 0; 1; 1; 1; 0; 1; 1; 1
Si Xi 6666
i 1
In order to illustrate the role of competition mechanism in the LCA algorithm, we compared the competition with the non-competition. In the latter m families were in the same position and each family mutated only a time. In this case, the LCA algorithm was similar to evolutionary strategy denoted by the term (1, 1)-ES (Nissen, 1994). The number of optimal solutions found was only 26 in 100 runs without the competition between families. Fig. 3 shows the processes of evolution for five families. Through the evolution of 30 generations, the initial values of objective function {6640, 6511, 6466, 6143, 6323} changed into {6665.4, 6661.4, 6661, 651.8, 6638}, none of which evolves to optimal solution.
Table 1 Statistical results obtained with different families (penalty coefficient a 1) m
N
Nf
CPU (s)
Successes
1 3 5 10 15
150 50 30 15 10
150 150 150 150 150
0.38 0.27 0.24 0.22 0.23
92 100 100 97 93
m , Number of families; N , number of generations; Nf, number of function evaluations (Nf m N ); CPU, computational time on 586 computer (150 MHz).
Fig. 3. The courses of evolution for five families in the case of noncompetition for solving knapsack problem.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
From two different processes of evolution in Figs. 2 and 3, it can be seen that the competition mechanism in the LCA algorithm plays a very important role. The introduction of competition mechanism improves largely the solution quality and advances the search efficiency.
4. Optimization of pressure relief header network
pipe segments and 34 fluid source nodes. The decision variables are the pipe diameters, which may take one of nine available discrete values for each segment. The number of possible unconstrained designs is 979, which is indeed a very large number. It is very difficult to use enumerate method for finding optimal solution because it is too time-consuming. The pressures in the upstream (A) and downstream (B) end of a pipe segment are approximated by: p2A p2B
4.1. Problem description In this section, we examine the problem of optimizing a pressure relief header network, which is used for a safety system in refineries and chemical plants. The role of the network is to transport fluids as expediently as possible from process equipment to the flare. In order to achieve sonic transport velocity through a relief valve for some transitory period, the pressure on downstream side of the valve must be remained below a certain value. Therefore, there is a maximum allowable pressure drop of each valve-flare path along the network, which constitutes a set of constraints for optimization of the problem. Fig. 4 is a pressure relief header network. Cheng and Mah (1976); Dolan et al. (1989) solved this problem with discrete merging method. Cardoso et al. (1994) applied SA to the problem. The network has 79 fixed-length
255
0:3697f (L Le )W 2 RT p2 Md 5
(4)
where L is the physical length of the pipe section; Le, the equivalent length; W , the mass flow rate; T , the temperature; M , the molecular weight; d , the pipe diameter; R , the gas constant; and f is a non-friction factor given by: f 0:0475
1:1744 10 12 W md
0:186
(5)
where m is fluid viscosity. The temperature and molecular weights corresponding to the various pipe sections are determined by a simple mixing rule (Dolan et al., 1989). The network cost is determined by the capital cost of the pipe segments. The objective is to minimize the total cost. The objective function is given by:
Fig. 4. Pressure relief header network.
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
256
min F
79 X
(abdi )Li
(6)
i 1
where a and b are constants, subject to the pressure constraints: X bi ] (p2A p2B ) i 1; 2; . . . ; 34 (7) where bi is the maximum allowable pressure drop between node i and the flare. 4.2. Problem solution To solve the problem with the LCA algorithm, we adopted the approach of Dolan et al. (1989) to construct mutation neighborhood, i.e. a pipe segment is selected randomly, and then its diameter is increased or decreased by 1 unit at the probability of 50%, respectively. In this way, the mutation of a time will not produce distinct change in the network cost, but the mutation of a few times leads possibly to notable change. Therefore, the above mutation neighborhood is to follow the basic principles of the LCA algorithm. The pressure constraints were handled by using the approach of penalty function. We used m families for representing m design configurations. In the progress of evolution, each family reproduced only one offspring in each generation. Initially each family had the same design configuration as that given in the literature (Dolan et al., 1989), where all pipe diameters were set to the largest possible value. Table 2 shows the statistical results obtained with 1, 5, 10 and 20 families, respectively, for 30 runs. Here, the same number of function evaluations was used for each different m for the purpose of comparison. Note that five families obtain the lowest cost and have the best robustness, with only a somewhat increases in computational time compared with 10 and 20 families. One family, corresponding to local search, is the worst and requires much more computational time than the multi-families. By comparison, we can state that the competition of multi-families increases significantly the solution quality and search
Fig. 5. The course of evolution for five families for optimizing pressure relief header network.
Table 3 Comparison of the LCA with DM and SA Algorithms a
DM SAb LCA a b
Nf
Average cost ($)
Best cost ($)
S.D.
/ 5 024 163 20 000
/ 165 353 164 210
165 018 165 018 163 970
/ 177 132
DM (Dolan et al., 1989). SA (Cardoso et al., 1994).
efficiency. Fig. 5 shows the course of evolution for five families. Table 3 gives the comparison of the LCA with discrete merge (DM; Dolan et al., 1989) and SA (Cardoso et al., 1994). The results show that the network cost of $163 970 obtained with the LCA is less than that of $165 018 with DM and SA, respectively. In addition, the LCA requires much fewer the number of function evaluations than SA, being no more than 0.4% of SA. Table 4 gives the best solution obtained, together with the solution of DM and SA.
Table 2 Statistical results obtained with different m families (a 7.1243 $/m; b 341.05 $/m2) m
N
Nf
CPU (s)
Average cost
Best cost ($)
S.D.
1 5 10 20
20 000 4000 2000 1000
20 000 20 000 20 000 20 000
405.6 263.6 247.6 249.4
166 540 164 210 164 250 164 632
165 230 163 970 164 010 164 280
1128 132 185 226
m , Number of families; N , number of generations; Nf, number of function evaluations (Nf m N ); CPU, computational time on 586 computer (150 MHz).
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
257
Table 4 Optimal solutions of DM, SA and LCA Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA) Branch Pipe (DM) Pipe (SA) Pipe (LCA)
1 1 1 1 13 2 2 2 25 1 1 1 37 2 2 2 49 2 2 2 61 1 1 1 73 1 3 1
2 2 2 2 14 2 2 2 26 1 1 1 38 3 3 2 50 3 3 3 62 4 4 3 74 3 3 3
3 2 2 2 15 2 2 2 27 1 1 1 39 2 2 2 51 2 2 2 63 2 2 2 75 1 1 1
4 1 1 1 16 2 2 2 28 1 1 1 40 3 3 3 52 3 3 3 64 2 2 2 76 4 4 5
5. Conclusions The framework of solving a combinatorial optimization problem with the LCA algorithm is proposed in this paper. The two basic principles determined mutation neighborhoods are given. The applications of the proposed algorithm to the knapsack problem and the optimization of pressure relief header network problem reveals that the LCA algorithm is able to find optimal or near-optimal solution after examining an extremely small fraction of search space. The main performance of the LCA algorithm is able to balance well local and global search. By ordering, in line-up the front families, which have obtained better solutions, are allocated fewer mutation times, which is favorable to speed up local search and accelerate finding optimal solution. The hind families, which have obtained worse solutions, are allocated more mutation times, which is helpful for global search. In particular, the LCA algorithm has the capability of overcoming local minimum. Even if all families gather round a local minimum, it can also surpass the local minimum. This is because the families of different positions in a line-up have the different mutation times, i.e. different search spaces. Owing to having larger search space, the hind families in the line-up likely find better solutions than this local minimum. Once the hind
5 2 2 2 17 1 2 1 29 2 2 2 41 1 1 1 53 2 2 2 65 2 2 2 77 2 2 2
6 1 1 1 18 1 2 1 30 1 2 1 42 3 3 3 54 2 2 2 66 3 3 2 78 3 3 3
7 1 1 1 19 1 1 1 31 2 2 2 43 4 4 4 55 2 2 2 67 2 2 1 79 7 7 7
8 1 1 1 20 1 1 1 32 1 1 1 44 4 6 4 56 3 3 2 68 3 3 3 / / / /
9 1 1 1 21 1 1 1 33 1 1 1 45 1 1 1 57 2 2 2 69 1 1 1 / / / /
10 1 1 1 22 1 3 1 34 1 1 1 46 2 1 1 58 4 4 3 70 4 3 3 / / / /
11 1 1 1 23 1 1 1 35 2 2 2 47 1 1 1 59 1 1 1 71 1 1 1 / / / /
12 1 1 1 24 1 1 1 36 2 2 2 48 2 2 2 60 4 4 3 72 3 3 3 / / / /
families escape the local minimum, the positions of these families will be moved to the front of the line-up. As a result, the original families in the front of the line-up will be moved to rear, which lets them obtain more mutation times, thus, they can also escape the local minimum. This characteristic embodies the unification of competition and cooperation between families. Because of the above characteristic, all families chase each other, and their positions in the line-up ascend alternately. Through competitions, the first family in the line-up is replaced continually by other families, or the value of objective function of the first position is updated continually. As a result, the optimal solution is approached rapidly.
References Androulakis, I. P., & Venkatasubramenian, V. (1991). A genetic algorithmic framework for process design and optimization. Computers and Chemical Engineering 15 , 217 /228. Cardoso, M. F., Salcedo, R. L., & de Feyo, A. (1994). Nonequilibrium simulated annealing: a faster approach to combinatorial minimization. Industrial Engineering and Chemical Research 33 , 1908 /1918. Cheng, W. B., & Mah, R. S. H. (1976). Optimal design of pressure relieving piping networks by discrete merging. American Institute of Chemical Engineering Journal 22 (3), 471 /476.
258
L. Yan / Computers and Chemical Engineering 27 (2003) 251 /258
Das, H., Cummings, P. T., & Levan, M. D. (1990). Scheduling of serial multiproduct batch processes via simulated annealing. Computers and Chemical Engineering 14 , 1351 /1362. Dolan, W. B., Cumings, P. T., & Levan, M. D. (1989). Process optimization via simulated annealing: application to network design. American Institute of Chemical Engineering Journal 35 , 725 /736. Floquet, P., Pibouleau, L., & Doemenech, S. (1994). Separation sequence synthesis: how to use simulated annealing procedure. Computers and Chemical Engineering 18 , 1141 /1148. Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning . Addison-Wesley. Greeff, D. J., & Aldrich, C. (1998). Empirical modeling of chemical process system with evolutionary programming. Computers and Chemical Engineering 22 , 995 /1005. Hanagandi, V., & Nikolaou, M. (1998). A hybrid approach to global optimization using a clustering algorithm in a genetic search framework. Computers and Chemical Engineering 22 , 1913 / 1925. Jung, J. H., Lee, C. H., & Lee, I. B. (1998). A genetic algorithm for scheduling of multi-product batch processes. Computers and Chemical Engineering 22 , 1725 /1730. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulation annealing. Science 220 , 671 /680.
Ku, H. M., & Karrimi, I. A. (1991). An evaluation of simulated annealing for batch process scheduling. Industrial Engineering and Chemical Research 30 , 163 /169. McKay, B., Willis, M., & Barton, G. (1997). Steady-state modeling of chemical process systems using genetic programming processes. Computers and Chemical Engineering 21 , 981 /996. Nissen, V. (1994). Solving the quadratic assignment problem with clues from nature. IEEE Transactions on Neural Network 5 , 66 /72. Papadimitriou, C. H., & Steiglitz, K. (1982). Combinatorial optimization algorithms and complexity . Printice-Hall. Patel, A. N., Mah, R. S. K., & Karlmi, I. A. (1991). Preliminary design of multiproduct non-continuous plants using simulated annealing. Computers and Chemical Engineering 15 , 451 /469. Wang, C., Quan, H., & Xu, X. (1995). Optimal design of multiproduct batch chemical process using genetic algorithms. Industrial Engineering and Chemical Research 35 , 3560 /3566. Yan, L. X. (1998). Ph.D. dissertation , Beijing: University of Chemical Technology. Yan, L. X., & Ma, D. X. (2001). Global optimization of nonconvex nonlinear programs using line-up competition algorithm. Computers and Chemical Engineering 25 , 1601 /1610. Yip, P., & Pao, Y. H. (1995). Combinatorial optimization with use of guided evolutionary simulated annealing. IEEE Transactions on Neural Networks 6 , 290 /295.