- Email: [email protected]
Incremental Beam search
Information Processing Letters 113 (2013) 888–893
Contents lists available at ScienceDirect
Information Processing Letters www.elsevier.com/locate/ipl
Incremental Beam search S.G. Vadlamudi a,∗ , Sandip Aine b , P.P. Chakrabarti a a b
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur, WB 721302, India Indraprastha Institute of Information Technology, New Delhi, DL 110020, India
a r t i c l e
i n f o
Article history: Received 9 September 2012 Received in revised form 29 August 2013 Accepted 29 August 2013 Available online 11 September 2013 Communicated by M. Chrobak Keywords: Algorithms Beam search Monotonicity Heuristic search Anytime algorithms
a b s t r a c t Beam search is a heuristic search algorithm that explores a state-space graph by expanding w most promising nodes at each level (depth) of the graph, where w is called the beamwidth which is taken as input from the user. The quality of the solution produced by beam search does not always monotonically improve with the increase in beam-width making it difficult to choose an appropriate beam-width for effective use. We present an algorithm called Incremental Beam Search (IncB) which guarantees monotonicity, and is also anytime in nature. Experimental results on the sliding-tile puzzle, the traveling salesman, and the single-machine scheduling problems show that IncB significantly outperforms basic monotonic methods such as iterative widening beam search as well as some of the state-ofthe-art anytime heuristic search algorithms in terms of the quality of the solution produced at the end as well as the anytime performance. © 2013 Elsevier B.V. All rights reserved.
1. Introduction Beam search [1,2] is one of the simplest and widely applied heuristic search techniques. It refers to a breadth-first search strategy which only stores a set of w most promising nodes at each level (where w is called the beamwidth), and prunes the remaining nodes. By restricting the node storage, the memory/time complexity of beam search, for a constant width w, becomes linear in the depth of search. Since beam search enforces unrestricted inadmissible pruning and has no provision for backtracking, it cannot guarantee completeness. However, its simplicity and effectiveness in various applications often make beam search a method of choice among heuristic search techniques, especially for very large state-space graphs. In this work, we address the monotonicity aspect of beam search. Although beam search produces solutions in bounded time and memory, the quality of the solutions may vary depending on the beam-width input. Typically, it is desired that the output solution quality increases with
*
Corresponding author. E-mail addresses: [email protected] (S.G. Vadlamudi), [email protected] (S. Aine), [email protected] (P.P. Chakrabarti). 0020-0190/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.ipl.2013.08.010
increase in beam-width since it corresponds to an increase in the explored portions of the graph and use of more resources in terms of memory and time. Such a property will help in choosing an appropriate beam-width easily based on the resources available while maximizing the solution quality. One can then simply choose the maximum beam-width that satisfies the memory and time constraints, which is in-turn computed based on the memory consumed per node, the branching factor of the graph, the rate of node expansion, and a reasonable bound on the depth of goal nodes. The possibility of non-monotonic progress of beam search is mentioned in [3,4]. A basic solution is proposed to address this problem in [4], where beam search is repeatedly run with increasing beam-widths starting from 1. This method is called as Iterative Widening Beam Search (IWB). A similar strategy can also be found with the name Iterative Broadening [5], though in a slightly different context. In [6], a theoretical analysis of beam search based on uniform cost tree model is presented which analytically demonstrates the possibility of highly non-monotonic behavior of beam search. Experimental observations on the non-monotonic nature of beam search in case of the Circular Packing Problem (CPP) are presented in [7], where
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
a monotonic beam search version called Adaptive Beam Search (ABS) is proposed. However, the extent of nonmonotonicity of beam search on various real application domains is not well studied. Also, while IWB re-expands a number of nodes, ABS focuses on less promising nodes as the search progresses, which point to the scope for improvement. In this paper, we study the performance of beam search in general as well as on a number of benchmarks from the Artificial Intelligence (AI) domain (Sliding-tile Puzzle Problem (SPP)) and the Optimization domain (Traveling Salesman Problem (TSP) and Single-Machine Scheduling problem (SMS)). The experiments highlight the often highly non-monotonic progress of beam search (Section 2). To address this problem, we introduce a new beam search based algorithm called Incremental Beam Search (IncB) which guarantees monotonicity (Section 3) and tries to proceed in a best-first manner without node re-expansions. Our objective here is also to develop anytime algorithms that can work on reasonably large sized problems. Experimental results on SPP, TSP and SMS show that IncB significantly outperforms monotonic methods such as IWB and ABS as well as some of the state-of-the-art anytime heuristic search algorithms like Beam-Stack search [8], AWA* [9], and ANA* [10] (Section 4). The performances of the algorithms are compared with respect to two metrics: 1) Quality of solution reported at the end in case of monotonic beam search methods, namely, IncB, IWB, and ABS, and 2) Anytime performance (solution quality with passing time). The quality of a solution is measured in terms of its closeness to the optimal solution (if known) or its closeness to the best known solution (if optimal solution is not known). Such a measure helps in normalizing the various results before the average is taken over all the instances, which gives a better picture of the relative performances than a simple direct average of the solution costs of various instances. 2. Beam Search observations We begin with an example (Fig. 1) which gives an insight as to how Beam Search with higher beam-width can get driven towards finding a poorer solution, as explained in the caption of the figure. Fig. 2 shows the results obtained when Beam Search is run with different beam-widths on various domains. Note
889
Fig. 1. A search tree of a minimization problem on which Beam Search with beam-width = 2 gives a poorer quality solution (length 9) than Beam search with beam-width = 1 (length 6). All nodes at depth 3 are goal nodes. Values inside brackets indicate the f-values of the corresponding nodes, with (3-depth) used as the heuristic estimate.
that, in all cases, high degrees of non-monotonicity are observed, where the output of Beam Search with a low beam-width is often much better than the output obtained with higher beam-widths. Clearly, choosing the optimal beam-width for which Beam Search gives the best possible solution using the available resources (time, memory) is very difficult with such highly non-monotonic behavior. 3. Incremental Beam Search (IncB) In this section, we present the proposed beam search method called Incremental Beam Search (IncB) (Algorithm 1) which does not re-expand nodes and produces solutions whose quality increases monotonically with increase in beam-width. The method partitions the available beam-width space into two regions, one is called the active beam-width region (ACT_BW), and the other is called the wait beamwidth region (WAIT_BW). At any point of time, the algorithm works as if it is performing Beam Search which only has the active beam-width region available to it. However, while deleting an unexpanded node from the active beam-width region due to active-beam-width cutoff, it judiciously uses the wait beam-width region to keep as many promising nodes as possible in the memory. Then, the algorithm iteratively increases the active beam-width from 1 to B (simultaneously decreasing the wait beamwidth) and thus works in an incremental manner. After each iteration, it brings the most promising node at each level from the wait beam-width region into the active
Fig. 2. Results of Beam Search algorithm for different beam-widths. Solution costs indicate the number of moves to reach the goal state in case of the first 24-puzzle from [11, Table II] (a), the length of the tour in case of gr202, a 202-city symmetric TSP from TSPLIB [12] (b), and the total early and tardy costs in case of a 100-job instance with tardy factor T = 0.6 and range factor R = 0.6 [13] (c).
890
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
Algorithm 1 Incremental Beam Search (IncB)
Algorithm 3 GenerateChildrenInc
1: INPUT :: A search graph G, a start node s, and the beam-width B. 2: for i = 0 to MAX_DEPTH − 1 do 3: OpenList (i ) ← φ ; ClosedList(i ) ← φ ; WaitList(i ) ← φ ; 4: BestSol ← ∞; g (s) ← 0; Calculate f (s); Level(s) ← 0; Insert s into OpenList(0); ACT_BW ← 1 (active beam-width); WAIT_BW ← B − ACT_BW; 5: for i = 0 to MAX_DEPTH − 1 do 6: while OpenList(i ) = φ do 7: n ← least f-valued node from OpenList(i ); 8: if IsGoal(n) then 9: if BestSol > f (n) then 10: BestSol ← f (n); Goal ← n; 11: Move n from OpenList (i ) to ClosedList(i ); 12: continue; 13: GenerateChildrenInc(n); Move n from OpenList (i ) to ClosedList(i ); 14: if WAIT_BW > 0 then 15: ACT_BW ← ACT_BW + 1; WAIT_BW ← WAIT_BW − 1; MoveOneColumnToActive(); 16: goto line 5 17: return BestSol, Goal;
1: INPUT :: Node n whose children are to be generated, ACT_BW, WAIT_BW, and the lists. 2: if Level(n) = MAX_DEPTH − 1 then 3: return; 4: for each successor n of n do 5: if n is not in OpenLists and ClosedLists then 6: Level(n ) ← Level(n) + 1; Insert n into OpenList(Level(n )); 7: else if g (n ) < its previous g-value then 8: Update Level(n ), g (n ), f (n ); Insert n into OpenList (Level(n )); 9: d ← Level(n) + 1; 10: if |OpenList(d)| + |ClosedList(d)| > ACT_BW then 11: if ClosedList(d) = φ then 12: m ← largest f-valued node from OpenList (d); Move m from OpenList(d) to WaitList(d); (insert) 13: if |WaitList(d)| > WAIT_BW then 14: Remove the largest f-valued node (highest indexed) from WaitList(d); 15: else 16: Remove the oldest node from ClosedList(d);
Algorithm 2 MoveOneColumnToActive 1: INPUT :: The lists. 2: for i = 0 to MAX_DEPTH − 1 do 3: if WaitList(i ) = φ then 4: n ← the least f-valued node (lowest indexed) from WaitList(i ); 5: if n is not in OpenLists and ClosedLists then 6: Move n from WaitList(i ) to OpenList (i ); 7: else if g (n) < g-value of existing node n then 8: Remove n ; Move n from WaitList(i ) to OpenList (i ); 9: else 10: Remove n;
beam-width region for its progress, instead of re-starting like IWB. The crucial part in the management of nodes in the WaitList at any level is to keep the nodes in sorted order as per their f-values (most promising ones first), and never allow any node to move from a higher indexed position to a lower indexed position. Otherwise, it would mean that the algorithm assumes that the WAIT_BW is large enough to have the node at the higher indexed position from where it is moved, whereas such a position does not exist for an instance of IncB where WAIT_BW is 1, thereby violating the incremental and monotonic nature of the algorithm. Note that, this management will result in getting the most promising node at each level from the wait region to the active region in each iteration, which helps the search to proceed efficiently as it expands the most promising nodes at each level in the active region. Here, MAX_DEPTH refers to the maximum depth of the search graph one wishes to explore for finding the solution. It is typically set by the user to a large value within which a solution is expected to be found. For graphs such as SPP, it corresponds to the maximum number of moves one is allowed to make to reach a goal node. Varying MAX_DEPTH in such cases may affect the performance of some algorithms. However, the variation of the performances are expected to be low when using large values of MAX_DEPTH. For graphs such as TSP and SMS, this cor-
responds to the depth of the goal nodes which is constant for a given instance. Lemma 1. IncB produces solutions whose quality increases monotonically with increasing beam-width. Proof. Let IncB 1 be an instance of IncB executing with beam-width b1 and IncB 2 be an instance of IncB executing with beam-width b2 (< b1 ). Since both the instances explore the same set of nodes in the same order until the b2 th iteration (by the aforesaid management of nodes in wait region and the MoveOneColumnToActive strategy after each iteration), both will produce the same solution at that point, where IncB 2 terminates. Proceeding further, IncB 1 only updates the current solution with a better solution. 2 Lemma 2. IncB with beam-width b uses at-most b nodes of memory at each level. Proof. Since ACT_BW + WAIT_BW = b is always satisfied, and lines 10 and 13 of GenerateChildrenInc routine ensure that all the generated nodes are restricted to this space. 2 Lemma 3. IncB with beam-width b expands at-most b × (b + 1)/2 nodes at each level. Proof. Since IncB expands at-most i nodes during ith iteration at each level (because OpenList at each level contains at-most i nodes as i is the active beam-width during ith iteration), with beam-width b it expands at-most 1 + 2 + · · · + b = b × (b + 1)/2 nodes at each level. 2 Note that, IncB, IWB, and ABS, all expand b × (b + 1)/2 nodes at each level on an average, while using the same amount of memory and guaranteeing monotonicity. While IWB re-expands a number of nodes, ABS appears to spend more efforts on relatively less promising nodes. On the other hand, IncB tries to explore the best possible (most promising) set of nodes without re-expansion.
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
891
Fig. 3. Comparison of the performance of IncB with those of the existing algorithms on various (sliding-tile) 24-puzzle instances. DFBB could not come up with any solution in most cases in the given time.
4. Experimental results Now, we present the results comparing the proposed monotonic beam search algorithm against several existing anytime algorithms, namely, Depth-first Branch and Bound (DFBB), Beam-Stack search [8], Anytime Window A* (AWA*) [9], Anytime Non-parametric A* (ANA*) [10], and IWB and ABS – which are also monotonic. Since the performance of Beam-Stack algorithm varies with the beamwidth, we have tested it with multiple values and presented the best results obtained amongst all of them. All the experiments have been performed on a Dell Precision T7600 Tower Workstation with Intel Xeon CPU E5-2687W at 3.1-GHz × 16 and 256-GB RAM. We display the results in terms of a metric called % Optimal Closeness (whenever optimal solution is known), which is defined as: % Optimal Closeness = (Optimal solution/Obtained solution) × 100. Sliding-tile Puzzle Problem (SPP). For our experiments, we have considered all 50 24-puzzle instances from [11, Table II]. Manhattan distance heuristic is used as the heuristic estimation function (which underestimates the actual distance to goal). All algorithms are allowed to explore up-to a maximum depth of 1000 (around 10 times the depth of optimal solutions). Comparison with ABS is not presented for Puzzle problem as it does not directly apply for beam-widths greater than the branching factor of the start node (which is at-most 4 in this case). The segmented approach taken in [7] for handling large beam-widths is not unique, and the performance of the algorithm varies with the kind of segmentation followed, which we do not explore in this paper. However, we do compare with ABS in case of TSP and SMS where the root nodes have a large branching factor. Fig. 3a presents the comparison of the quality of solutions reported at the termination by the beam search algorithms IncB and IWB (both of which have same memory requirement and expand similar number of nodes). Clearly, performance of IncB is superior. Fig. 3b presents the comparison of anytime performances of various algo-
rithms with the proposed algorithm. BS-x denotes the instance of Beam-Stack search run with beam-width x. IncB and IWB are given large enough beam-widths such that they can run for the entire duration of 64 minutes. BeamStack search is given up-to a beam-width where the time taken for producing the initial solution is just more than that of the other algorithms. Note that, IncB outperforms the other algorithms. Traveling Salesman Problem (TSP). We chose the first 50 symmetric TSPs (when sorted in increasing order of their sizes) from the Traveling Salesman Problem Library (TSPLIB) [12] for our experiments. These range from burma14 to gr202 where the numerical postfixes denote the size of the TSPs. Minimum Spanning Tree (MST) heuristic is used as the heuristic estimation function (which is an under-estimating heuristic). Fig. 4a shows the comparison of the quality of solution reported at the termination of the beam search algorithms, namely, IncB, IWB, and ABS (all of which have same memory requirement and expand similar number of nodes). Once again, IncB outperforms the other algorithms. Fig. 4b shows the comparison of anytime performances of various algorithms. It can be noted that the more the beamwidth given to beam-stack search the more is the delay in producing the initial solution which hampers its anytime performance (the larger beam-widths are given in the first place to check whether its performance will improve later on during the time window, which did not happen). IncB continues to give better performance than the others. Single-Machine Scheduling Problem (SMS). We have generated 48 SMS problem instances1 with different tardy factors and range factors using similar settings as given in [13]. Each instance contains 100 jobs. For each job j, an integer processing time p j , earliness penalty h j , and tardiness penalty w j are generated from the uniform distribution [1, 10]. Let P be the sum of processing times of all the 1 The SMS benchmarks used in this paper are available at https://sites. google.com/site/satyagautamv/resources/sms-benchmarks.
892
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
Fig. 4. Comparison of the performance of IncB with those of the existing algorithms on various TSP instances.
Fig. 5. Comparison of the performance of IncB with those of the existing algorithms on various SMS problem instances. ANA* could not come up with any solution in most cases in the given time.
jobs. For each job j, an integer due date is generated from the uniform distribution [ P (1 − T − R /2), P (1 − T + R /2)], where T is the tardy factor, set at 0.0, 0.2, 0.4, 0.6, 0.8 and 1.0, and R is the range factor, set at 0.2, 0.4, 0.6 and 0.8. For each combination of parameters, two problems are generated. The lower bound computation method proposed in [13] is used for computing the under-estimating heuristic over the unscheduled jobs. Here, since the optimal solutions are not available, we display the results in terms of a metric called % Comparative Closeness (measures closeness to the best known solution), which is defined as: % Comparative Closeness = (Best Known solution/ Obtained solution) × 100. We obtained the best known solution for each instance by finding the best result produced amongst all the algorithms for that instance. Fig. 5a shows the comparison of endtime performances of IncB, IWB, and ABS for different beam-widths on the 48 SMS problem instances. Note that, IncB outperforms IWB and is competitive with ABS during the initial portion before outperforming it as well. The performance of ABS does not improve after a while as its larger beam-width executions focus on less promising nodes. Fig. 5b presents
the comparison of anytime performances of IncB with several other existing algorithms on the SMS instances. Here, AWA* and ANA* have consumed up-to 200 GB of memory individually per instance (other algorithms consumed only a few MB), whereas their performances are not impressive. We observe that the anytime performance of IncB is superior to that of others except for a minute period in the beginning. In conclusion, we observed that beam search is highly non-monotonic in terms of its solution quality with respect to its beam-width across various application domains. We presented a monotonic beam search algorithm called Incremental Beam Search which produces better solutions than several existing algorithms at endtime as well as anytime, and that can handle large sized problem instances under limited memory conditions. Acknowledgement S.G. Vadlamudi thanks SAP Labs India for their support through Doctoral Fellowship.
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
References [1] B. Lowerre, The harpy speech recognition system, PhD thesis, Carnegie Mellon University. [2] R. Bisiani, Beam search, in: Encyclopedia of Artificial Intelligence, 1987, pp. 56–58. [3] L.-C. Chu, B.W. Wah, Band search: An efficient alternative to guided depth-first search, in: ICTAI, 1992, pp. 154–161. [4] P. Norvig, Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp, 1st edition, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1992. [5] M.L. Ginsberg, W.D. Harvey, Iterative broadening, Artif. Intell. 55 (1992) 367–383. [6] S. Aine, P.P. Chakrabarti, An analysis of breadth-first beam search using uniform cost trees, in: ISAIM, 2010. [7] H. Akeb, M. Hifi, R. M’Hallah, A non-oscillating beam-search with a look-ahead for the circular packing problem, in: IEEE International
893
Conference on Industrial Engineering and Engineering Management, 2009, pp. 365–369. [8] R. Zhou, E.A. Hansen, Beam-stack search: Integrating backtracking with beam search, in: Proceedings of the 15th International Conference on Automated Planning and Scheduling (ICAPS-05), 2005, pp. 90–98. [9] S. Aine, P.P. Chakrabarti, R. Kumar, AWA* – A window constrained anytime heuristic search algorithm, in: M.M. Veloso (Ed.), IJCAI, 2007, pp. 2250–2255. [10] J. van den Berg, R. Shah, A. Huang, K.Y. Goldberg, Anytime nonparametric A*, in: AAAI, 2011, pp. 105–111. [11] R.E. Korf, A. Felner, Disjoint pattern database heuristics, Artif. Intell. 134 (1–2) (2002) 9–22. [12] G. Reinelt, TSPLIB – A traveling salesman problem library, ORSA J. Comput. 3 (1991) 376–384. [13] G. Li, Single machine earliness and tardiness scheduling, Eur. J. Oper. Res. 96 (3) (1997) 546–558.
Contents lists available at ScienceDirect
Information Processing Letters www.elsevier.com/locate/ipl
Incremental Beam search S.G. Vadlamudi a,∗ , Sandip Aine b , P.P. Chakrabarti a a b
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur, WB 721302, India Indraprastha Institute of Information Technology, New Delhi, DL 110020, India
a r t i c l e
i n f o
Article history: Received 9 September 2012 Received in revised form 29 August 2013 Accepted 29 August 2013 Available online 11 September 2013 Communicated by M. Chrobak Keywords: Algorithms Beam search Monotonicity Heuristic search Anytime algorithms
a b s t r a c t Beam search is a heuristic search algorithm that explores a state-space graph by expanding w most promising nodes at each level (depth) of the graph, where w is called the beamwidth which is taken as input from the user. The quality of the solution produced by beam search does not always monotonically improve with the increase in beam-width making it difficult to choose an appropriate beam-width for effective use. We present an algorithm called Incremental Beam Search (IncB) which guarantees monotonicity, and is also anytime in nature. Experimental results on the sliding-tile puzzle, the traveling salesman, and the single-machine scheduling problems show that IncB significantly outperforms basic monotonic methods such as iterative widening beam search as well as some of the state-ofthe-art anytime heuristic search algorithms in terms of the quality of the solution produced at the end as well as the anytime performance. © 2013 Elsevier B.V. All rights reserved.
1. Introduction Beam search [1,2] is one of the simplest and widely applied heuristic search techniques. It refers to a breadth-first search strategy which only stores a set of w most promising nodes at each level (where w is called the beamwidth), and prunes the remaining nodes. By restricting the node storage, the memory/time complexity of beam search, for a constant width w, becomes linear in the depth of search. Since beam search enforces unrestricted inadmissible pruning and has no provision for backtracking, it cannot guarantee completeness. However, its simplicity and effectiveness in various applications often make beam search a method of choice among heuristic search techniques, especially for very large state-space graphs. In this work, we address the monotonicity aspect of beam search. Although beam search produces solutions in bounded time and memory, the quality of the solutions may vary depending on the beam-width input. Typically, it is desired that the output solution quality increases with
*
Corresponding author. E-mail addresses: [email protected] (S.G. Vadlamudi), [email protected] (S. Aine), [email protected] (P.P. Chakrabarti). 0020-0190/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.ipl.2013.08.010
increase in beam-width since it corresponds to an increase in the explored portions of the graph and use of more resources in terms of memory and time. Such a property will help in choosing an appropriate beam-width easily based on the resources available while maximizing the solution quality. One can then simply choose the maximum beam-width that satisfies the memory and time constraints, which is in-turn computed based on the memory consumed per node, the branching factor of the graph, the rate of node expansion, and a reasonable bound on the depth of goal nodes. The possibility of non-monotonic progress of beam search is mentioned in [3,4]. A basic solution is proposed to address this problem in [4], where beam search is repeatedly run with increasing beam-widths starting from 1. This method is called as Iterative Widening Beam Search (IWB). A similar strategy can also be found with the name Iterative Broadening [5], though in a slightly different context. In [6], a theoretical analysis of beam search based on uniform cost tree model is presented which analytically demonstrates the possibility of highly non-monotonic behavior of beam search. Experimental observations on the non-monotonic nature of beam search in case of the Circular Packing Problem (CPP) are presented in [7], where
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
a monotonic beam search version called Adaptive Beam Search (ABS) is proposed. However, the extent of nonmonotonicity of beam search on various real application domains is not well studied. Also, while IWB re-expands a number of nodes, ABS focuses on less promising nodes as the search progresses, which point to the scope for improvement. In this paper, we study the performance of beam search in general as well as on a number of benchmarks from the Artificial Intelligence (AI) domain (Sliding-tile Puzzle Problem (SPP)) and the Optimization domain (Traveling Salesman Problem (TSP) and Single-Machine Scheduling problem (SMS)). The experiments highlight the often highly non-monotonic progress of beam search (Section 2). To address this problem, we introduce a new beam search based algorithm called Incremental Beam Search (IncB) which guarantees monotonicity (Section 3) and tries to proceed in a best-first manner without node re-expansions. Our objective here is also to develop anytime algorithms that can work on reasonably large sized problems. Experimental results on SPP, TSP and SMS show that IncB significantly outperforms monotonic methods such as IWB and ABS as well as some of the state-of-the-art anytime heuristic search algorithms like Beam-Stack search [8], AWA* [9], and ANA* [10] (Section 4). The performances of the algorithms are compared with respect to two metrics: 1) Quality of solution reported at the end in case of monotonic beam search methods, namely, IncB, IWB, and ABS, and 2) Anytime performance (solution quality with passing time). The quality of a solution is measured in terms of its closeness to the optimal solution (if known) or its closeness to the best known solution (if optimal solution is not known). Such a measure helps in normalizing the various results before the average is taken over all the instances, which gives a better picture of the relative performances than a simple direct average of the solution costs of various instances. 2. Beam Search observations We begin with an example (Fig. 1) which gives an insight as to how Beam Search with higher beam-width can get driven towards finding a poorer solution, as explained in the caption of the figure. Fig. 2 shows the results obtained when Beam Search is run with different beam-widths on various domains. Note
889
Fig. 1. A search tree of a minimization problem on which Beam Search with beam-width = 2 gives a poorer quality solution (length 9) than Beam search with beam-width = 1 (length 6). All nodes at depth 3 are goal nodes. Values inside brackets indicate the f-values of the corresponding nodes, with (3-depth) used as the heuristic estimate.
that, in all cases, high degrees of non-monotonicity are observed, where the output of Beam Search with a low beam-width is often much better than the output obtained with higher beam-widths. Clearly, choosing the optimal beam-width for which Beam Search gives the best possible solution using the available resources (time, memory) is very difficult with such highly non-monotonic behavior. 3. Incremental Beam Search (IncB) In this section, we present the proposed beam search method called Incremental Beam Search (IncB) (Algorithm 1) which does not re-expand nodes and produces solutions whose quality increases monotonically with increase in beam-width. The method partitions the available beam-width space into two regions, one is called the active beam-width region (ACT_BW), and the other is called the wait beamwidth region (WAIT_BW). At any point of time, the algorithm works as if it is performing Beam Search which only has the active beam-width region available to it. However, while deleting an unexpanded node from the active beam-width region due to active-beam-width cutoff, it judiciously uses the wait beam-width region to keep as many promising nodes as possible in the memory. Then, the algorithm iteratively increases the active beam-width from 1 to B (simultaneously decreasing the wait beamwidth) and thus works in an incremental manner. After each iteration, it brings the most promising node at each level from the wait beam-width region into the active
Fig. 2. Results of Beam Search algorithm for different beam-widths. Solution costs indicate the number of moves to reach the goal state in case of the first 24-puzzle from [11, Table II] (a), the length of the tour in case of gr202, a 202-city symmetric TSP from TSPLIB [12] (b), and the total early and tardy costs in case of a 100-job instance with tardy factor T = 0.6 and range factor R = 0.6 [13] (c).
890
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
Algorithm 1 Incremental Beam Search (IncB)
Algorithm 3 GenerateChildrenInc
1: INPUT :: A search graph G, a start node s, and the beam-width B. 2: for i = 0 to MAX_DEPTH − 1 do 3: OpenList (i ) ← φ ; ClosedList(i ) ← φ ; WaitList(i ) ← φ ; 4: BestSol ← ∞; g (s) ← 0; Calculate f (s); Level(s) ← 0; Insert s into OpenList(0); ACT_BW ← 1 (active beam-width); WAIT_BW ← B − ACT_BW; 5: for i = 0 to MAX_DEPTH − 1 do 6: while OpenList(i ) = φ do 7: n ← least f-valued node from OpenList(i ); 8: if IsGoal(n) then 9: if BestSol > f (n) then 10: BestSol ← f (n); Goal ← n; 11: Move n from OpenList (i ) to ClosedList(i ); 12: continue; 13: GenerateChildrenInc(n); Move n from OpenList (i ) to ClosedList(i ); 14: if WAIT_BW > 0 then 15: ACT_BW ← ACT_BW + 1; WAIT_BW ← WAIT_BW − 1; MoveOneColumnToActive(); 16: goto line 5 17: return BestSol, Goal;
1: INPUT :: Node n whose children are to be generated, ACT_BW, WAIT_BW, and the lists. 2: if Level(n) = MAX_DEPTH − 1 then 3: return; 4: for each successor n of n do 5: if n is not in OpenLists and ClosedLists then 6: Level(n ) ← Level(n) + 1; Insert n into OpenList(Level(n )); 7: else if g (n ) < its previous g-value then 8: Update Level(n ), g (n ), f (n ); Insert n into OpenList (Level(n )); 9: d ← Level(n) + 1; 10: if |OpenList(d)| + |ClosedList(d)| > ACT_BW then 11: if ClosedList(d) = φ then 12: m ← largest f-valued node from OpenList (d); Move m from OpenList(d) to WaitList(d); (insert) 13: if |WaitList(d)| > WAIT_BW then 14: Remove the largest f-valued node (highest indexed) from WaitList(d); 15: else 16: Remove the oldest node from ClosedList(d);
Algorithm 2 MoveOneColumnToActive 1: INPUT :: The lists. 2: for i = 0 to MAX_DEPTH − 1 do 3: if WaitList(i ) = φ then 4: n ← the least f-valued node (lowest indexed) from WaitList(i ); 5: if n is not in OpenLists and ClosedLists then 6: Move n from WaitList(i ) to OpenList (i ); 7: else if g (n) < g-value of existing node n then 8: Remove n ; Move n from WaitList(i ) to OpenList (i ); 9: else 10: Remove n;
beam-width region for its progress, instead of re-starting like IWB. The crucial part in the management of nodes in the WaitList at any level is to keep the nodes in sorted order as per their f-values (most promising ones first), and never allow any node to move from a higher indexed position to a lower indexed position. Otherwise, it would mean that the algorithm assumes that the WAIT_BW is large enough to have the node at the higher indexed position from where it is moved, whereas such a position does not exist for an instance of IncB where WAIT_BW is 1, thereby violating the incremental and monotonic nature of the algorithm. Note that, this management will result in getting the most promising node at each level from the wait region to the active region in each iteration, which helps the search to proceed efficiently as it expands the most promising nodes at each level in the active region. Here, MAX_DEPTH refers to the maximum depth of the search graph one wishes to explore for finding the solution. It is typically set by the user to a large value within which a solution is expected to be found. For graphs such as SPP, it corresponds to the maximum number of moves one is allowed to make to reach a goal node. Varying MAX_DEPTH in such cases may affect the performance of some algorithms. However, the variation of the performances are expected to be low when using large values of MAX_DEPTH. For graphs such as TSP and SMS, this cor-
responds to the depth of the goal nodes which is constant for a given instance. Lemma 1. IncB produces solutions whose quality increases monotonically with increasing beam-width. Proof. Let IncB 1 be an instance of IncB executing with beam-width b1 and IncB 2 be an instance of IncB executing with beam-width b2 (< b1 ). Since both the instances explore the same set of nodes in the same order until the b2 th iteration (by the aforesaid management of nodes in wait region and the MoveOneColumnToActive strategy after each iteration), both will produce the same solution at that point, where IncB 2 terminates. Proceeding further, IncB 1 only updates the current solution with a better solution. 2 Lemma 2. IncB with beam-width b uses at-most b nodes of memory at each level. Proof. Since ACT_BW + WAIT_BW = b is always satisfied, and lines 10 and 13 of GenerateChildrenInc routine ensure that all the generated nodes are restricted to this space. 2 Lemma 3. IncB with beam-width b expands at-most b × (b + 1)/2 nodes at each level. Proof. Since IncB expands at-most i nodes during ith iteration at each level (because OpenList at each level contains at-most i nodes as i is the active beam-width during ith iteration), with beam-width b it expands at-most 1 + 2 + · · · + b = b × (b + 1)/2 nodes at each level. 2 Note that, IncB, IWB, and ABS, all expand b × (b + 1)/2 nodes at each level on an average, while using the same amount of memory and guaranteeing monotonicity. While IWB re-expands a number of nodes, ABS appears to spend more efforts on relatively less promising nodes. On the other hand, IncB tries to explore the best possible (most promising) set of nodes without re-expansion.
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
891
Fig. 3. Comparison of the performance of IncB with those of the existing algorithms on various (sliding-tile) 24-puzzle instances. DFBB could not come up with any solution in most cases in the given time.
4. Experimental results Now, we present the results comparing the proposed monotonic beam search algorithm against several existing anytime algorithms, namely, Depth-first Branch and Bound (DFBB), Beam-Stack search [8], Anytime Window A* (AWA*) [9], Anytime Non-parametric A* (ANA*) [10], and IWB and ABS – which are also monotonic. Since the performance of Beam-Stack algorithm varies with the beamwidth, we have tested it with multiple values and presented the best results obtained amongst all of them. All the experiments have been performed on a Dell Precision T7600 Tower Workstation with Intel Xeon CPU E5-2687W at 3.1-GHz × 16 and 256-GB RAM. We display the results in terms of a metric called % Optimal Closeness (whenever optimal solution is known), which is defined as: % Optimal Closeness = (Optimal solution/Obtained solution) × 100. Sliding-tile Puzzle Problem (SPP). For our experiments, we have considered all 50 24-puzzle instances from [11, Table II]. Manhattan distance heuristic is used as the heuristic estimation function (which underestimates the actual distance to goal). All algorithms are allowed to explore up-to a maximum depth of 1000 (around 10 times the depth of optimal solutions). Comparison with ABS is not presented for Puzzle problem as it does not directly apply for beam-widths greater than the branching factor of the start node (which is at-most 4 in this case). The segmented approach taken in [7] for handling large beam-widths is not unique, and the performance of the algorithm varies with the kind of segmentation followed, which we do not explore in this paper. However, we do compare with ABS in case of TSP and SMS where the root nodes have a large branching factor. Fig. 3a presents the comparison of the quality of solutions reported at the termination by the beam search algorithms IncB and IWB (both of which have same memory requirement and expand similar number of nodes). Clearly, performance of IncB is superior. Fig. 3b presents the comparison of anytime performances of various algo-
rithms with the proposed algorithm. BS-x denotes the instance of Beam-Stack search run with beam-width x. IncB and IWB are given large enough beam-widths such that they can run for the entire duration of 64 minutes. BeamStack search is given up-to a beam-width where the time taken for producing the initial solution is just more than that of the other algorithms. Note that, IncB outperforms the other algorithms. Traveling Salesman Problem (TSP). We chose the first 50 symmetric TSPs (when sorted in increasing order of their sizes) from the Traveling Salesman Problem Library (TSPLIB) [12] for our experiments. These range from burma14 to gr202 where the numerical postfixes denote the size of the TSPs. Minimum Spanning Tree (MST) heuristic is used as the heuristic estimation function (which is an under-estimating heuristic). Fig. 4a shows the comparison of the quality of solution reported at the termination of the beam search algorithms, namely, IncB, IWB, and ABS (all of which have same memory requirement and expand similar number of nodes). Once again, IncB outperforms the other algorithms. Fig. 4b shows the comparison of anytime performances of various algorithms. It can be noted that the more the beamwidth given to beam-stack search the more is the delay in producing the initial solution which hampers its anytime performance (the larger beam-widths are given in the first place to check whether its performance will improve later on during the time window, which did not happen). IncB continues to give better performance than the others. Single-Machine Scheduling Problem (SMS). We have generated 48 SMS problem instances1 with different tardy factors and range factors using similar settings as given in [13]. Each instance contains 100 jobs. For each job j, an integer processing time p j , earliness penalty h j , and tardiness penalty w j are generated from the uniform distribution [1, 10]. Let P be the sum of processing times of all the 1 The SMS benchmarks used in this paper are available at https://sites. google.com/site/satyagautamv/resources/sms-benchmarks.
892
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
Fig. 4. Comparison of the performance of IncB with those of the existing algorithms on various TSP instances.
Fig. 5. Comparison of the performance of IncB with those of the existing algorithms on various SMS problem instances. ANA* could not come up with any solution in most cases in the given time.
jobs. For each job j, an integer due date is generated from the uniform distribution [ P (1 − T − R /2), P (1 − T + R /2)], where T is the tardy factor, set at 0.0, 0.2, 0.4, 0.6, 0.8 and 1.0, and R is the range factor, set at 0.2, 0.4, 0.6 and 0.8. For each combination of parameters, two problems are generated. The lower bound computation method proposed in [13] is used for computing the under-estimating heuristic over the unscheduled jobs. Here, since the optimal solutions are not available, we display the results in terms of a metric called % Comparative Closeness (measures closeness to the best known solution), which is defined as: % Comparative Closeness = (Best Known solution/ Obtained solution) × 100. We obtained the best known solution for each instance by finding the best result produced amongst all the algorithms for that instance. Fig. 5a shows the comparison of endtime performances of IncB, IWB, and ABS for different beam-widths on the 48 SMS problem instances. Note that, IncB outperforms IWB and is competitive with ABS during the initial portion before outperforming it as well. The performance of ABS does not improve after a while as its larger beam-width executions focus on less promising nodes. Fig. 5b presents
the comparison of anytime performances of IncB with several other existing algorithms on the SMS instances. Here, AWA* and ANA* have consumed up-to 200 GB of memory individually per instance (other algorithms consumed only a few MB), whereas their performances are not impressive. We observe that the anytime performance of IncB is superior to that of others except for a minute period in the beginning. In conclusion, we observed that beam search is highly non-monotonic in terms of its solution quality with respect to its beam-width across various application domains. We presented a monotonic beam search algorithm called Incremental Beam Search which produces better solutions than several existing algorithms at endtime as well as anytime, and that can handle large sized problem instances under limited memory conditions. Acknowledgement S.G. Vadlamudi thanks SAP Labs India for their support through Doctoral Fellowship.
S.G. Vadlamudi et al. / Information Processing Letters 113 (2013) 888–893
References [1] B. Lowerre, The harpy speech recognition system, PhD thesis, Carnegie Mellon University. [2] R. Bisiani, Beam search, in: Encyclopedia of Artificial Intelligence, 1987, pp. 56–58. [3] L.-C. Chu, B.W. Wah, Band search: An efficient alternative to guided depth-first search, in: ICTAI, 1992, pp. 154–161. [4] P. Norvig, Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp, 1st edition, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1992. [5] M.L. Ginsberg, W.D. Harvey, Iterative broadening, Artif. Intell. 55 (1992) 367–383. [6] S. Aine, P.P. Chakrabarti, An analysis of breadth-first beam search using uniform cost trees, in: ISAIM, 2010. [7] H. Akeb, M. Hifi, R. M’Hallah, A non-oscillating beam-search with a look-ahead for the circular packing problem, in: IEEE International
893
Conference on Industrial Engineering and Engineering Management, 2009, pp. 365–369. [8] R. Zhou, E.A. Hansen, Beam-stack search: Integrating backtracking with beam search, in: Proceedings of the 15th International Conference on Automated Planning and Scheduling (ICAPS-05), 2005, pp. 90–98. [9] S. Aine, P.P. Chakrabarti, R. Kumar, AWA* – A window constrained anytime heuristic search algorithm, in: M.M. Veloso (Ed.), IJCAI, 2007, pp. 2250–2255. [10] J. van den Berg, R. Shah, A. Huang, K.Y. Goldberg, Anytime nonparametric A*, in: AAAI, 2011, pp. 105–111. [11] R.E. Korf, A. Felner, Disjoint pattern database heuristics, Artif. Intell. 134 (1–2) (2002) 9–22. [12] G. Reinelt, TSPLIB – A traveling salesman problem library, ORSA J. Comput. 3 (1991) 376–384. [13] G. Li, Single machine earliness and tardiness scheduling, Eur. J. Oper. Res. 96 (3) (1997) 546–558.