- Email: [email protected]
Minimum-weight spanning tree algorithms A survey and empirical study
Computers & Operations Research 28 (2001) 767}785
Minimum-weight spanning tree algorithms A survey and empirical study CuK neyt F. Bazlamac7 cm , Khalil S. Hindi * Electrical and Electronics Engineering Department, Middle East Technical University, Ankara 06531, Turkey Department of Systems Engineering, Brunel University, Uxbridge, Middlesex UB8 3PH, UK Received 1 August 1999; received in revised form 1 December 1999
Abstract The minimum-weight spanning tree problem is one of the most typical and well-known problems of combinatorial optimisation. E$cient solution techniques had been known for many years. However, in the last two decades asymptotically faster algorithms have been invented. Each new algorithm brought the time bound one step closer to linearity and "nally Karger, Klein and Tarjan proposed the only known expected linear-time method. Modern algorithms make use of more advanced data structures and appear to be more complicated to implement. Most authors and practitioners refer to these but still use the classical ones, which are considerably simpler but asymptotically slower. The paper "rst presents a survey of the classical methods and the more recent algorithmic developments. Modern algorithms are then compared with the classical ones and their relative performance is evaluated through extensive empirical tests, using reasonably large-size problem instances. Randomly generated problem instances used in the tests range from small networks having 512 nodes and 1024 edges to quite large ones with 16 384 nodes and 524 288 edges. The purpose of the comparative study is to investigate the conjecture that modern algorithms are also easy to apply and have constants of proportionality small enough to make them competitive in practice with the older ones. Scope and purpose The minimum-weight spanning tree (MST) problem is a well-known combinatorial optimisation problem concerned with "nding a spanning tree of an undirected, connected graph, such that the sum of the weights of the selected edges is minimum. The importance of this problem derives from its direct applications in the design of computer, communication, transportation, power and piping networks; from its appearance as part of solution methods to other problems to which it applies less directly such as network reliability, clustering and classi"cation problems and from its occurrence as a subproblem in the solution of other problems like the travelling salesman problem, the multi-terminal #ow problem, the matching problem and the capacitated
* Corresponding author. Tel.: #1895-203299; fax: #171-691-9454. E-mail address: [email protected] (K.S. Hindi). 0305-0548/01/$ - see front matter 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 3 0 5 - 0 5 4 8 ( 0 0 ) 0 0 0 0 7 - 1
768
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
MST problem. Although e$cient solution techniques capable of solving large instances had existed, there has been sustained e!ort over the last two decades to invent asymptotically faster algorithms. With each new algorithm found the time bound approached linearity. Finally, an expected linear-time method has been proposed. The purpose of this work is to survey the classical and modern solution techniques and empirically compare the performance of the existing methods. 2001 Elsevier Science Ltd. All rights reserved.
Keywords: Network optimisation; Graph algorithms; Minimum spanning tree; Linear-time algorithms; Performance evaluation
1. Introduction The minimum-weight spanning tree problem (MSTP), one of the most typical and well-known problems of combinatorial optimisation, is that of "nding a spanning tree of an undirected, connected graph, such that the sum of the weights of the selected edges is minimum. MSTP is important and very popular for the following reasons: 1. It has direct applications in the design of computer and communication networks, power and leased-line telephone networks, wiring connections, links in a transportation network, piping in a #ow network, etc. 2. It o!ers a method of solution to other problems to which it applies less directly, such as network reliability, clustering and classi"cation problems. 3. It often occurs as a subproblem in the solution of other problems. For example, minimum spanning tree (MST) algorithms are used in several exact and approximation algorithms for the travelling salesman problem, the multi-terminal #ow problem, the matching problem and the capacitated MST problem. 4. E$cient solution techniques capable of solving large instances exist. The long and rich history of the problem is detailed in the surveys by Pierce [1], Ma$oli [2] and Graham and Hell [3]. Ma$oli's survey takes a broader view and classi"es various forms of optimum undirected tree problems, with emphasis on their computational complexity. The survey of Graham and Hell concentrates solely on the MST; not only surveying the algorithms but also tracing their independent sources. However, this informative survey stops at 1985 emphasising classical algorithms mainly. In the last two decades, e!orts were concentrated on "nding ever faster algorithms, making use of more modern data structures, with each new algorithm bringing the time bound a step closer to linearity. Finally, Karger et al. [4] proposed the only known linear expected-time algorithm for the restricted random access computation model. This is a randomised, recursive algorithm and requires the solution of a related problem, that of verifying whether a given spanning tree is minimum. Numerous authors have studied di!erent forms of the classical algorithms and proposed e$cient implementation techniques (see for example [5]). Reports on computational experiments of various types have also appeared [6,7].
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
769
The continuing interest and developments in the "eld and the discovery of recent algorithms, including the randomised linear-time algorithm of Karger et al. [4], argue for a comprehensive review of the existing methods, classical and modern, and for a comparative empirical study. Most of the modern algorithms are not di$cult to implement. Yet, it is almost standard practice among authors and practitioners to refer to these, but still use one of the classical algorithms which are considerably simpler, though asymptotically slower. The present work aims to verify empirically the conjecture that modern algorithms are also easy to use and have constants of proportionality small enough to make them competitive in practice with older algorithms. In the next two sections, the formal de"nition of the problem and a general solution technique are given. Section 4 is an overview of the classical methods and Section 5 includes a survey of the more recent algorithmic developments. Section 6 empirically evaluates the comparative performance of modern and classical MST algorithms and investigates the feasibility of using the recent methods in solving large problem instances. Section 7 presents a summary and the conclusions.
2. Problem de5nition Given an undirected graph G"(<, E), where < denotes the set of vertices with n"< and E the set of edges with m"E and a real number w(e) for each edge e3E called the weight of edge e, the MSTP is formally de"ned as "nding a spanning tree ¹H on G, such that w(¹H)"min w(e) is 2 CZ2 the minimum taken over all possible spanning trees of G. Given a graph G, a spanning tree ¹ and a cotree edge e"i, j3E!¹, the unique cycle in G consisting of the edge e and the edges of the unique chain in ¹ between i and j is called a fundamental cycle of G relative to ¹ with respect to e. For a graph G and two distinct vertices say i and j, let X be any subset of vertices that contains i but not j and let XM be its complement (i.e., XM "<X), then set X is a cut and the set of edges (i, j) i3X, j3XM , (i, j)3E is called a cutset. The removal of the arcs in a cutset leaves a disconnected subgraph of G. Corresponding to each edge e of a spanning tree ¹ of a connected graph G, there is a unique cutset called the fundamental cutset of ¹ with respect to edge e. The two theorems on the optimality conditions for a MST are stated below without proofs. Two elegant proofs can be found in [8,9]. Theorem 1 (Fundamental cutset optimality). A spanning tree T in a weighted graph is a MST if and only if every edge in the tree is a minimum-weight edge in the fundamental cutset dexned by that edge. Theorem 2 (Fundamental cycle optimality). A spanning tree T in a graph G is a MST if and only if every edge e3E!¹ is a maximum weight edge in the unique fundamental cycle dexned by that edge.
3. General solution The MSTP is solved by a simple incremental, greedy method: the MST is built edge by edge, including appropriate small edges and excluding appropriate large ones. The technique is greedy in the sense that at every stage, the best possible edge is chosen for inclusion in the MST without
770
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
producing a cycle in the subgraph constructed so far or for exclusion from the MST without disconnecting the graph. In his monograph, Tarjan [10] represents the construction process as one of edge colouring. Starting from an initial uncoloured edge set, we colour one edge at a time either blue (included) or red (excluded) according to the following two rules: Blue rule: Select a cutset that does not contain a blue edge. Among the uncoloured edges in the cutset, select one of minimum weight and colour it blue. Red rule: Select a simple cycle containing no red edges. Among the uncoloured edges on the cycle, select one of maximum weight and colour it red. The above blue and red rules are direct applications of the fundamental cutset and fundamental cycle optimality theorems, respectively. Either rule can be applied at any time, making the method non-deterministic. The most important property of the greedy method is that it colours all the edges of any connected graph and maintains a MST containing all of the blue edges and none of the red ones. In the next two sections an overview of the classical methods and a survey of the more recent algorithmic developments are given. Table 1 summarises and presents the existing MST algorithms Table 1 Minimal spanning tree algorithms Algorithm
Special data structures and subproblems used
Time complexity
Boruvka [11] Kruskal (sorted) [12]
O(m log n) O(m(m, n))
Kruskal (not sorted) [12]
Disjoint set union algorithm Disjoint set union (with path compression and merging with rank) Disjoint set union and heapsort
Prim [13]
}
O(n)
Prim Prim Prim
Binary heap d-Heap F-Heap
O(m log n) O(nd log n#m log n) B B O(n log n#m)
Yao [15] Cheriton and Tarjan [16]
Heaps of size k, a selection algorithm Doubly linked queue, leftist heap with lazy meld and lazy deletion, disjoint set union
O(m log log n) O(m log log n)
Fredman and Tarjan [17] Gabow et al. [18]
F-heap, doubly linked queue F-heaps with packets, disjoint set union
O(m(m, n)) O(m log (m, n))
Karger [22] Karger et al. [4]
Randomisation Randomisation, recursion, a linear-time veri"cation algorithm
O(n log n#m) O(m)
(m, n): Inverse Ackermann's function. (m, n)"mini logG n)m/n and log n"n. Expected running time.
O(m log n)
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
771
in "ve categories: classical algorithms (Boruvka, Kruskal, Prim); Prim implementations using di!erent forms of heap structure; algorithms which introduced for the "rst time the special use of heaps and obtained faster running times than the classical algorithms (Yao, Cheriton and Tarjan); algorithms which used F-heaps for the "rst time (Fredman and Tarjan, Gabow et al.); and the more recent algorithms which use randomisation (Karger, Karger et al.). For each algorithm, all the special structures used and the corresponding time complexities are also included.
4. Classical algorithms There are three classical algorithms; namely, Boruvka [10,11], Kruskal [12] and Prim [13]; the last two of which are presented in almost every well-known text in the "eld. All three algorithms use the blue rule. Although algorithms that use the red rule can be designed, until recently, they were thought to be less e$cient. The algorithm of Karger et al. [4] uses the blue rule in conjunction with the red. The blue edges form a forest, consisting of a set of trees that are called blue trees. The most general blue rule algorithm begins with an initial set of blue trees and applies the following colouring step n!1 times: Colouring step: (General) Select a blue tree; "nd a minimum-weight edge incident to this tree and colour it blue. The three classical algorithms di!er in their starting states and colouring steps; in other words, they di!er in the criterion used to select the next edge or edges to be added in each iteration. A brief summary of the three classical algorithms is given below with the conventions of Tarjan [10]. The Boruvka algorithm: Start with the initial set of n blue trees, each consisting of one vertex and no edges, and repeat the following step until there is only one blue tree. Colouring step: For every blue tree ¹, select a minimum weight edge incident to ¹. Colour all selected edges blue. The algorithm builds the trees uniformly throughout the graph. It is, therefore, suitable for use in parallel computations. The Kruskal algorithm: Sort the edges in the order of increasing weights and apply the following step to the edges in the sorted list until the number of blue edges is n!1. Colouring step: If the edge considered has both endpoints in the same blue tree, colour it red; otherwise colour it blue. The algorithm builds up blue trees in an irregular fashion dictated only by edge weights. It is worth noting that the algorithm is best in situations where the edges are given in sorted order or can be sorted fast (like when the weights are small integers, making it possible to employ radix sort) or where the graph is sparse. The Prim algorithm: Use an arbitrary starting vertex s and apply the following step n!1 times. Colouring step: Let ¹ be the blue tree containing s. Select a minimum weight edge incident to ¹ and colour it blue. The algorithm generates only one nontrivial blue tree from a single root. The Boruvka algorithm runs in O(m log n) time (see Table 1). If edges are in disorder with respect to their weight, the Kruskal algorithm requires O(m log n) time. Given a sorted edge list, "nding a MST requires O(m(m, n)) time, assuming the use of the disjoint set union algorithm of Tarjan
772
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
[10], where (m, n) is the inverse Ackermann's function. (m, n) is de"ned as (m, n)"mini*1 A(i, W m/n X )'log n for m*n*1 where Ackermann's function A(i, j) for i, j*1 is given by A(1, j)"2H for j*1, A(i, 1)"A(i!1, 2) for i*2, A(i, j)"A(i!1, A(i, j!1)) for i, j*2. The function A(i, j) grows exponentially fast and hence (m, n) is a very slowly growing function. (m, n))3 for n(2. Hence, for all practical purposes (m, n) is assumed to be a constant not larger than four. The Prim algorithm with an implementation using d-heaps, as proposed by Johnson [14], runs in O(nd log n#m log n). In the case of binary heaps (i.e., d"2), the time complexity is O(m log n). B B Choosing d"U2#m/n V , the time bound becomes O(m log n). If m"(n>C), for some >K L positive , the running time is O(m/). Thus, the algorithm with Johnson's implementation is well suited for dense graphs as well as sparse ones and the method is asymptotically worse than that of Kruskal only if the edges are pre-sorted.
5. Modern algorithms Renewed interest in the MSTP in the last two decades led to faster algorithms. Yao [15] was the "rst to discover an implementation of Boruvka which had a faster asymptotic running time. Cheriton and Tarjan [16] followed by proposing a similar but slightly more practical algorithm. Both algorithms are based on the general blue rule algorithm given above. They both use heaps to implement e$cient Boruvka-like colouring steps and run in O(m log log n) time. With the invention of an e$cient form of heap, called the Fibonacci heap (F-heap), Fredman and Tarjan [17] developed an implementation of the MST algorithm which runs in O(m(m, n)), where (m, n)"mini logGn)m/n with logn"n. In addition, they pointed out that the use of an F-heap in its simplest form in the implementation of the Prim algorithm, i.e., replacing the binary or d-heap with an F-heap, solves the MSTP in O(n log n#m) time. Gabow et al. [18] introduced the use of packets in conjunction with F-heaps and modi"ed the Fredman and Tarjan algorithm to achieve the slightly better time complexity of O(m log (m, n)). The method of Gabow et al. was the fastest theoretically until recently. Currently, the linear expected-time algorithm of Karger et al. [4] stands as theoretically the fastest. It is a randomised and recursive algorithm which also requires the solution of a related problem, that of verifying whether a given spanning tree is minimum. This subproblem itself has been studied earlier, but it was the linear veri"cation algorithm invented by Dixon et al. [19] which made the proof of the existence of a linear expected time MST algorithm possible. The O(m)-time veri"cation algorithm of King [20] seems to be much simpler. In the following, we brie#y describe one example for each of the last three groups in Table 1. The modern algorithms chosen and used in this work are either the fastest or the most practical in each group and are described along with implementation details in [21].
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
773
The Cheriton and Tarjan Algorithm A heap is kept for each blue tree. Each heap holds the edges with at least one endpoint in the tree and which are candidates for becoming blue; the cost of an edge being its key in the heap. The major di!erence from the Boruvka algorithm is that in each iteration we pick one blue tree in the forest for the merge operation. In conjunction with the heap, the selection rule for picking a blue tree for merging plays an important role in beating the O(m log n) time bound. Two alternatives are [16]: E pick uniformly the "rst in a queue of candidate blue trees (implementation with a doubly linked list is su$cient to have an O(1) time operation including deletion from the queue), E pick the smallest candidate tree (less e$cient with an O(n) implementation). This algorithm is asymptotically faster than any of the three classical algorithms for sparse graphs, but is slower by a factor of O(log log n) than the Prim algorithm for dense graphs. The Fredman and Tarjan Algorithm The invention of Fibonacci heaps (F-heaps) by Fredman and Tarjan [17] opened the way to improving some network optimisation algorithms, including those for the shortest path and the MST problems. The most important property of F-heaps is that arbitrary deletion from an n-item heap is possible in O(log n) amortised time, while all other standard heap operations require O(1) amortised time. In many network optimisation algorithms, the number of deletions is relatively small compared to the total number of heap operations. Thus F-heaps provide a very e$cient tool for obtaining asymptotically faster algorithms. The simplest and most straightforward use of an F-heap in the context of the MSTP is in the implementation of the Prim algorithm to "nd the minimum weight edge incident to the blue tree containing the starting vertex. The algorithm requires n deletemin operations and O(m) other heap operations, none of them deletions. Therefore, using F-heaps leads to O(n log n#m) running time (Table 1). The O(m(m, n)) time bound of Fredman and Tarjan is due to a di!erent strategy, which grows a single tree only until its heap of neighbouring vertices exceeds a certain critical size. Then, another tree starting from a new vertex is grown until the heap gets large again. Continuing in this way, the stopping criterion is that every vertex should be in a tree. The algorithm consists of one or more passes of the above form. Each pass implicitly considers the trees of the forest obtained in the previous pass as super vertices. These super vertices are then grown into larger trees in the next pass. After a su$cient number of passes, only the MST remains. Fredman and Tarjan's implementation uses a clean-up operation to achieve condensing the graph before starting each pass. This requires the numbering of the trees consecutively from one and assigning to each vertex the number of the tree containing it. The clean-up requires, a two-pass radix sort for the lexicographic ordering of the edges on the numbers of the trees containing their endpoints, the scanning of the edge list to save the appropriate ones, and the construction for each old tree t of a list of the edges with one endpoint in t after the clean-up. The method of Gabow et al. [18] is an improved version of the Fredman and Tarjan algorithm. The main idea is to group the edges with a common vertex into &packets' and to work only with packet minima. The structure is similar to that of Fredman and Tarjan in general, but additional
774
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
F-heaps are used to handle the packets and to "nd the packet minima. The packets are initialised according to parameter packet size. In order to achieve the claimed time bound O(m log (m, n)), packet size should be chosen as (m, n). Moreover, the heap size selected for the ith pass is chosen as k "2K L and k "2IG\ for i*2. G For practical problems, even if the network is very sparse, packet size is usually small. Hence, the advantage of using packets and using F-heaps for such a small number of elements in each packet is not very big. Therefore, we feel that for most realistic problem sizes, the Fredman and Tarjan algorithm will be comparable to, or even better than, that of Gabow et al. However, this conjecture remains to be tested. The Algorithm of Karger et al. The algorithm of Karger et al. solves the more general problem of "nding a minimum forest in a possibly disconnected graph, but we assume the graph to be connected for simplicity. We also assume that all edge weights are distinct, since an ordering is possible even if they are not. Unlike most algorithms for "nding a MST, the Karger et al. algorithm makes use of both the blue and red rules. The algorithm relies on a random sampling step to discard edges that cannot be in the MST. We need the following terminology to state the result that indicates the e!ectiveness of this edge elimination step. Given an edge (x, y)3E and a forest F in G, F(x, y) denotes the path connecting x and y in F with w (x, y) denoting the maximum weight of an edge on F(x, y). We use the $ convention w (x, y)"R if x and y are not connected in F. An edge is F if w(x, y)'w (x, y) $ $ and F otherwise. It is worth noting that all edges of F are F and no F edges can be in the MST of G as a result of the red rule. There exist O(m)-time algorithms to compute the F edges (for example, [19,20]). Lemma 1 (Karger et al. [4]). Let H be a subgraph obtained from G by including each edge independently with probability p, and let F be the minimum spanning forest of H. The expected number of F edges in G is at most n/p where n is the number of vertices. We omit the proof and only state that the lemma generalises to matroids. There are three major operations, namely the Boruvka step, the random sampling step and the veri"cation step: Boruvka step: Given a graph G, apply the Boruvka algorithm to carry out one colouring step only. Contract the graph G, i.e., replace each blue tree by a single vertex, delete edges whose endpoints are the same after the replacement and delete all but the lowest-weight edge among each set of multiple edges, again after the replacement. Random sampling step: Given a contracted graph G, choose a subgraph H by selecting each edge in G independently with a probability 1/2. Verixcation step: Given any minimum spanning forest F of H-G, "nd all the F edges (both those in H and not in H) and delete them from G to reduce the graph further Each Boruvka step reduces the number of vertices by at least one half. The following algorithm is recursive and generates two subproblems after each execution of step 1.
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
775
Algorithm 1. Given G"(<, E), apply two successive Boruvka steps to the graph to contract G. 2. Apply the random sampling step to the contracted graph to select H. 3. Apply the algorithm recursively producing a minimum spanning forest F of the H formed in step 2. 4. Given F of H, apply the veri"cation step to the subgraph H, which was chosen, and obtain a graph G which is reduced further. 5. Apply the algorithm recursively to G to compute the minimum spanning forest F of G . 6. Return those edges contracted in step 1 together with the edges of F . The worst case and the expected running time of the minimum spanning forest algorithm is O(minn, m log n) and O(m), respectively. It runs in O(m) time with probability 1!exp(!(m)) [4].
6. Computational results Computational testing of the surveyed algorithms was carried out on problem instances of various sizes. All implementations were in Pascal and computations were carried out on an HP735 workstation running under UNIX. To the best knowledge of the authors, this is the "rst report in the literature on a comparative, empirical study of the performance of the algorithms of Boruvka and Karger et al. on problem instances of considerable size. The main objectives of the computational study are to: E Compare the performance of the classical and modern algorithms and investigate empirically the practicability of adopting the recently developed ones among the latter to substitute the former. E Observe the e!ect of varying network sizes and compare the real time performance of the algorithms with the theoretical bounds. E Pinpoint the fastest MST algorithm in practice; at least for the range of problem sizes tested. Random problem instances were generated. The size of the largest problem instances was chosen such that they are solvable on a relatively small computer. The numbers of nodes, ranged, in powers of two, from 512 (0.5K) to 16K. The large resulting networks were chosen to be sparse; with m/n ratios of two, four, eight, sixteen and thirty two. Each n;m combination constitutes a group of "ve di!erent problem instances; all the timings are averages for each group. For each problem instance, nodes were selected randomly from a 1000;1000 grid with a uniform probability density. To ensure connectivity, edges (i, j) j"i#1 were chosen "rst, i.e., each node i was connected to node i#1, then the end nodes of the remaining edges were selected from the complete graph at random. The cost of an edge was calculated as the integer part of the Euclidean distance between its two endpoints. Each problem instance was solved to "nd the corresponding MST using the di!erent algorithms. Dynamic memory management techniques are employed to make handling large networks possible. Since memory management operations are machine and compiler dependent, we consider
776
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
them as general utility operations and do not include their execution times in the following discussion. Results regarding the Kruskal algorithm in relation to sorting methods can be analysed in two ways. One is by keeping n constant and increasing m or vice versa; the other is by increasing the input size while keeping the m/n ratio constant. For the latter, we double both input parameters n and m. It is worth noting that the density of the graph decreases with such an approach. It is well known that none of the existing sorting algorithms is the best all round. Some are good for small m, others for large m; where m is the number of items in the list to be sorted. Insertion sort works well when the list is already partially ordered. Merge sort has the best worst-case behaviour, but it requires more storage than a heap sort. Quick sort has the best average behaviour, but its worst-case behaviour is O(m). In this work, we employed both the quick sort and the heap sort in computing the MST with the Kruskal algorithm. We analyse the results by keeping n constant "rst. Even though it requires some extra memory, quick sort is usually recommended for large m. The results obtained in our tests in sorting rather large lists demonstrate that it is fast and advantageous up to m"16K, but at m"32K, quick sort and heap sort become comparable. Beyond this point, we observe the non-linear character of quick sort and heap sort dominates. Fig. 1 illustrates these thresholds for n"2K. Fig. 2 on the other hand demonstrates the non-linearity of quick sort and the superiority of heap sort for n"8K. Although heap sort is also non-linear, its non-linearity was not observed within the problem range under consideration. The conjecture that heap sort is usually not quite as fast as quick sort does not
Fig. 1. Performance of Kruskal and the sorting algorithms used for n"2K.
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
777
Fig. 2. Performance of Kruskal and the sorting algorithms used for n"8K.
hold for very large m. Once the edges are available in sorted order, the average computing time for Kruskal is extremely small (see Figs. 1 and 2). The second type of analysis, in which the m/n ratio is kept constant while increasing the input size, produced similar results. The gap between quick sort and heap sort increases as the m/n ratio increases for large n. The Prim algorithm performs better with a conventional binary heap than with any other heap. In general, the implementation with d-heaps comes second, but the implementation with F-heaps gains an advantage over it as the number of edges increases (see Table 2). Increasing m/n decreases the di!erence between the F-heap and d-heap implementations, and the implementation with F-heap dominates the one with d-heap for m/n"32, except for small n (n)2K). It is worth noting that implementing the Prim algorithm without the use of heaps is extremely ine$cient comparatively (see column no-hp in Table 2). The results also reveal the quadratic nature of the algorithm. The Boruvka algorithm can be implemented in various ways. In the current study, we used the disjoint set-union algorithm with and without clean-up, i.e., permanently deleting the edges that have both end-points in the same connected component. Figs. 3 and 4 are performance plots. An interesting observation is that the use of clean-up turns out to be counter productive. In our implementation, we "rst form a new edge list in the form of a doubly linked list, so that each time an edge is found to have two end points in the same subtree, it can be deleted permanently from the edge list in O(1) time. However, the overhead of the initial copying process and accessing the edges
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
778
Table 2 Performance of Prim's algorithm using di!erent forms of heaps (s) m
n"0.5K
n"1K
n"2K
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
1K 2K 4K 8K 16K 32K 64K
0.08 0.09 0.09 0.11 0.14 * *
0.01 0.01 0.02 0.03 0.07 * *
0.01 0.01 0.02 0.04 0.06 * *
0.01 0.02 0.03 0.04 0.08 * *
* 0.37 0.38 0.40 0.43 0.48 *
* 0.03 0.03 0.05 0.08 0.15 *
* 0.03 0.03 0.05 0.08 0.13 *
* 0.03 0.04 0.06 0.10 0.16 *
* * 1.88 1.90 1.93 2.00 2.16
* * 0.06 0.08 0.11 0.19 0.34
* * 0.06 0.07 0.11 0.18 0.31
* * 0.06 0.09 0.13 0.21 0.37
m
n"4K
8K 16K 32K 64K 128K 256K
n"8K
n"16K
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
12.69 12.73 12.80 12.94 13.25 *
0.14 0.19 0.27 0.44 0.81 *
0.14 0.18 0.26 0.41 0.73 *
0.15 0.21 0.30 0.47 0.81 *
* 54.38 54.46 54.68 55.09 55.75
* 0.33 0.42 0.61 0.99 1.80
* 0.30 0.40 0.57 0.91 1.60
* 0.33 0.46 0.66 1.03 1.75
* * 220.41 220.51 221.86 222.03
* * 0.71 0.92 1.32 2.15
* * 0.66 0.88 1.26 1.98
* * 0.70 0.97 1.40 2.15
no-hp: without any heap; d-hp: with d-heap; b-hp: with binary heap; F-hp: with Fibonacci heap.
in the list is larger than the gain from reducing the number of edges. Although the problems considered are quite large, they require few Boruvka phases and the reduction in the total number of edges does not compensate for the list processing overhead. Nevertheless, in much larger problems, Boruvka's algorithm with clean-up may dominate. Next, we compare one representative of each group of modern algorithms in Table 1. Cheriton and Tarjan, Fredman and Tarjan and Karger et al. are chosen for their practicality or recency as representatives of each group. Figs. 5}8 present their performances, respectively. The Cheriton and Tarjan algorithm is relatively the simplest and the most e$cient in terms of real CPU time for the problem instances considered. This result is veri"ed up to n"16K and m"256K. Due to physical memory limitations, the observed non-linearity beyond n"16K and m"128K was not veri"ed using even larger problem instances. The Fredman and Tarjan algorithm seems to be most interesting from the viewpoint of practical performance. Although it is fast in theory, the break even point seems far away. Moreover, unless n is su$ciently large, it is highly sensitive to the m/n ratio. The sparser the graph, the more ine$cient is the method, which, at least for the range of problem sizes we have used, con#icts with the asymptotic bound. Figs. 6 and 7 are performance plots which illustrate that given n, increasing m/n decreases the running time until the point where the heap size k is larger than n and hence where only one pass is performed. The choice of the parameter k dictates the number of passes. Smaller values of k reduce the time per pass; larger values reduce the number of passes. In other
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
Fig. 3. Performance of Boruvka's algorithm.
Fig. 4. Performance of Boruvka's algorithm w.r.t. to m/n ratio.
779
780
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
Fig. 5. Performance of Cheriton and Tarjan's algorithm.
Fig. 6. Performance of Fredman and Tarjan's algorithm.
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
Fig. 7. Performance of Fredman and Tarjan's algorithm w.r.t. m/n.
Fig. 8. Performance of Karger et al.'s algorithm.
781
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
782
words, the trees that are grown without interruption tend to be small in passes with a small k. Practical results demonstrate that stopping the tree growing process and restarting growing a new tree from scratch very often, following the necessary heap initialisation and other assignments, is ine$cient. Choosing k"2K R, where t is the number of trees before the pass, optimises the running time per pass to be O(m), but this is not observed for inputs which are not su$ciently large. The ine$ciency for small m/n is also partly due to the use of the clean-up process after each pass. Unless n and m are su$ciently large, each additional pass adds a substantial amount of clean-up time. The algorithm of Karger et al. is also not practical for the problem sizes we considered. Due to its recursive nature, we store a reduced network in each subproblem and, hence, spend more time in copying and processing these local networks. Such overheads are either not small or even inordinate. Moreover, it is important to note that the expected time linearity of the algorithm is not observed with the data set used. Table 3 o!ers a comparison of the modern algorithms with the classical for n"8K and n"16K, with m/n ranging between 2 and 32. The results indicate that the conventional binary heap implementation of the Prim algorithm is the fastest for the problem sizes considered. Table 4 indicates roughly the amount of work one has to do to implement each algorithm. The algorithm of Karger et al. is the most di$cult to implement as it also requires the solution of the minimum spanning tree veri"cation and the least-common ancestor subproblems in linear time. A problem related to that addressed in this work; namely, the problem of re-optimisation of solutions, is noteworthy. This problem, also known in the literature as the problem of MST maintenance on dynamic graphs, deals with maintaining a minimum spanning tree on a graph subject to modi"cations (change of edge weights, insertion and deletion of edges and vertices). Neither classical nor modern algorithms provide adequate data structures and mechanisms for fast maintenance. However, there are for this purpose fast methods that use special structures and
Table 3 Performance of MST algorithms (s) n
m
Kruskal
Prim
Boruvka
Cheriton
4K
8K 16K 32K 64K 128K
0.21 0.36 0.66 1.31 2.69
0.14 0.18 0.26 0.41 0.73
0.24 0.44 0.84 1.62 3.39
0.44 0.65 0.96 1.61 2.86
7.53 6.78 0.84 1.44 2.64
1.36 2.19 4.02 7.64 18.77
8K
16K 32K 64K 128K 256K
0.46 0.78 1.44 2.83 5.58
0.30 0.40 0.57 0.91 1.60
0.54 0.99 1.83 3.83 7.42
0.94 1.31 1.99 3.42 5.90
29.91 25.82 1.78 3.06 5.58
2.96 5.17 10.47 26.58
With conventional binary heap. Not performed due to memory limitations.
Fredman
Karger
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
783
Table 4 Code sizes of MST algorithms (bytes) Code
Kruskal
Prim
Boruvka
Cheriton
Fredman
Karger
Source Executable
10664 7168
8788 6656
12456 7680
25761 13312
35742 15872
82121 38656
With conventional binary heap.
techniques (see for example [23]). A brief survey of the MST maintenance problem can be found in [21].
7. Conclusions The paper brie#y surveys the existing algorithms for solving the MSTP and classi"es them according to their similarities or the tools and subproblems they use. The feasibility of using the modern methods, which are more complicated compared to the classical ones, has been investigated. The empirical results lead to the conclusion that while the algorithms developed in recent years are theoretically very important, they require excessive problem sizes to justify their use as an alternative to the classical methods. The problem dimensions we have experimented with are rather large, but still not su$ciently large to make the asymptotically fast algorithms competitive in practice with the older ones. Among modern methods, the Cheriton and Tarjan round robin algorithm is relatively the simplest. It has constants of proportionality small enough to make it the most e$cient and most competitive of the modern methods. This work has helped in observing the e!ect of varying network sizes and in comparing the empirical time performance of the algorithms with the theoretical bounds. In the case of the Fredman and Tarjan algorithm, it is found that extremely large problem sizes are required to reach the break even point and hence the method is not viable to apply in realistic cases. The expected linear-time algorithm of Karger et al. was also found to be not feasible to use, at least for the range of problem sizes we considered. It uses the complicated subproblem of verifying a MST in linear time, which in turn requires the solution of the nearest common ancestor problem in linear time. For the range of problem sizes considered here, the Prim algorithm with a conventional binary heap implementation is found to be the fastest. In view of its simplicity and practicality, it is safe to conclude that the Prim algorithm will continue to serve e$ciently with a conventional binary heap implementation in most cases and possibly with a F-heap implementation in problem instances much larger than those considered in this study.
References [1] Pierce AR. Bibliography on algorithms for shortest path, shortest spanning tree and related circuit routing problems (1956}1974). Networks 1975;5:129}49.
784
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
[2] Ma$oli F. Complexity of optimum undirected tree problems: a survey of recent results. In: Ausiello G, Lucertini M, editors. Analysis and design of algorithms in combinatorial optimization. International Center for Mechanical Sciences. CISM Courses and Lectures, 266. New York: Springer, 1981. p. 107}28. [3] Graham RL, Hell P. On the history of the minimum spanning tree problem. Annals of the History of Computing 1985;7:43}57. [4] Karger DR, Klein PN, Tarjan RE. A randomized linear-time algorithm to "nd minimum spanning trees. Journal of the Association for Computing Machinery 1995;42/2:321}8. [5] Haymond RE, Jarvis JP, Shier DR. Computational methods for minimum spanning tree algorithms. SIAM Journal on Scienti"c and Statistical Computing 1984;5/1:157}74. [6] Glover F, Klingman D, Krishnan R, Padman A. An in-depth empirical investigation of non-greedy approaches for the minimum spanning tree problem. European Journal of Operational Research 1992;56:343}56. [7] Moret BME, Shapiro D. How to "nd a minimum spanning tree in practice. In: Maurer H, editor. New results and new trends in computer science: proceedings, Graz, Austria, June 1991. Lecture Notes in Computer Science, Vol. 555. Berlin: Springer, 1991. p. 192}203. [8] Balakrishnan VK. Network optimization. London: Chapman and Hall, 1995. [9] Gondran M, Minoux M. Graphs and Algorithms. New York: Wiley, 1984. [10] Tarjan RE. Data structures and network algorithms. In CBMS-NFS Regional Conference Series in Applied Mathematics. Philadelphia: Society for Industrial and Applied Mathematics, 1983. p. 44. [11] Boruvka O. O jisteH m probleH mu minimaH lnmH m. PraH ca MoravskeH Pr\ mH rodove\ deckeH Spolec\ nosti 1926;3:37}58 [in Czech], cited in [3,10]. [12] Kruskal JB. On the shortest spanning subtree of a graph and the traveling salesman problem. Proceedings of the American Mathematical Society 1956;7:48}50. [13] Prim RC. Shortest connection networks and some generalizations. Bell System Technical Journal 1957;36:1389}401. [14] Johnson DB. Priority queues with update and "nding minimum spanning trees. Information Processing Letters 1975;4:53}7. [15] Yao A. An O(Elog log <) algorithm for "nding minimum spanning trees. Information Processing Letters 1975;4:21}3. [16] Cheriton D, Tarjan RE. Finding minimum spanning trees. SIAM Journal on Computing 1976;5:724}42. [17] Fredman ML, Tarjan RE. Fibonacci heaps and their uses in improved network optimisation algorithms. Journal of the Association for Computing Machinery 1987;34/3:596}615. [18] Gabow HN, Galil Z, Spencer TH, Tarjan RE. E$cient algorithms for "nding minimum spanning trees in undirected and directed graphs. Combinatorica 1986;6:109}22. [19] Dixon B, Rauch M, Tarjan RE. Veri"cation and sensitivity analysis of minimum spanning trees in linear time. SIAM Journal on Computing 1992;21:1184}92. [20] King V. A simpler minimum spanning tree veri"cation algorithm. In: Akl SG, Dehne F. editors. Algorithms and data structures: Proceedings of the 4th International Workshop, Kingston, Canada. Lecture Notes in Computer Science, 955. Berlin: Springer, 1995. p. 440}8. [21] Bazlamac7 cm CF. Optimised network design: minimum spanning trees and minimum concave-cost problems. Ph.D. thesis, University of Manchester, Institute of Science and Technology, Manchester, England, 1996. [22] Karger DR. Random sampling in matroids, with applications to graph connectivity and minimum spanning trees. In Proceedings of the 34th Annual IEEE Symposium on Foundations of Computer Science. Los Alamitos, California: IEEE Computer Society Press, 1993. pp. 84}93. [23] Eppstein D, Italiano GF, Tamassia R, Tarjan RE, Westbrook J, Yung M. Maintenance of a minimum spanning forest in a dynamic plane graph. Journal of Algorithms 1992;13:33}54.
CuK neyt F. Bazlamac7 c1 is an Assistant Professor in Electrical and Electronics Engineering Department, Middle East Technical University (METU), Ankara, Turkey. He received his B.S. and M.S. degrees, both in electrical and electronics engineering, from the Middle East Technical University and his Ph.D. degree in Computation from the University of Manchester, Institute of Science and Technology (UMIST), Manchester. His current research interests include network
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
785
and graph algorithms, synthesis and design of computer, communication and transportation networks, applications of graph theory in operations research, metaheuristics and advanced data structures and algorithms. Khalil S. Hindi is a Professor of Systems Engineering at Brunel University, Greater London. His current research interests are in computer-aided management, planning, operation, scheduling and control of engineering systems; particularly manufacturing systems, electric power systems and gas and water distribution and transmission systems, employing computing and mathematical modelling techniques.
Minimum-weight spanning tree algorithms A survey and empirical study CuK neyt F. Bazlamac7 cm , Khalil S. Hindi * Electrical and Electronics Engineering Department, Middle East Technical University, Ankara 06531, Turkey Department of Systems Engineering, Brunel University, Uxbridge, Middlesex UB8 3PH, UK Received 1 August 1999; received in revised form 1 December 1999
Abstract The minimum-weight spanning tree problem is one of the most typical and well-known problems of combinatorial optimisation. E$cient solution techniques had been known for many years. However, in the last two decades asymptotically faster algorithms have been invented. Each new algorithm brought the time bound one step closer to linearity and "nally Karger, Klein and Tarjan proposed the only known expected linear-time method. Modern algorithms make use of more advanced data structures and appear to be more complicated to implement. Most authors and practitioners refer to these but still use the classical ones, which are considerably simpler but asymptotically slower. The paper "rst presents a survey of the classical methods and the more recent algorithmic developments. Modern algorithms are then compared with the classical ones and their relative performance is evaluated through extensive empirical tests, using reasonably large-size problem instances. Randomly generated problem instances used in the tests range from small networks having 512 nodes and 1024 edges to quite large ones with 16 384 nodes and 524 288 edges. The purpose of the comparative study is to investigate the conjecture that modern algorithms are also easy to apply and have constants of proportionality small enough to make them competitive in practice with the older ones. Scope and purpose The minimum-weight spanning tree (MST) problem is a well-known combinatorial optimisation problem concerned with "nding a spanning tree of an undirected, connected graph, such that the sum of the weights of the selected edges is minimum. The importance of this problem derives from its direct applications in the design of computer, communication, transportation, power and piping networks; from its appearance as part of solution methods to other problems to which it applies less directly such as network reliability, clustering and classi"cation problems and from its occurrence as a subproblem in the solution of other problems like the travelling salesman problem, the multi-terminal #ow problem, the matching problem and the capacitated
* Corresponding author. Tel.: #1895-203299; fax: #171-691-9454. E-mail address: [email protected] (K.S. Hindi). 0305-0548/01/$ - see front matter 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 3 0 5 - 0 5 4 8 ( 0 0 ) 0 0 0 0 7 - 1
768
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
MST problem. Although e$cient solution techniques capable of solving large instances had existed, there has been sustained e!ort over the last two decades to invent asymptotically faster algorithms. With each new algorithm found the time bound approached linearity. Finally, an expected linear-time method has been proposed. The purpose of this work is to survey the classical and modern solution techniques and empirically compare the performance of the existing methods. 2001 Elsevier Science Ltd. All rights reserved.
Keywords: Network optimisation; Graph algorithms; Minimum spanning tree; Linear-time algorithms; Performance evaluation
1. Introduction The minimum-weight spanning tree problem (MSTP), one of the most typical and well-known problems of combinatorial optimisation, is that of "nding a spanning tree of an undirected, connected graph, such that the sum of the weights of the selected edges is minimum. MSTP is important and very popular for the following reasons: 1. It has direct applications in the design of computer and communication networks, power and leased-line telephone networks, wiring connections, links in a transportation network, piping in a #ow network, etc. 2. It o!ers a method of solution to other problems to which it applies less directly, such as network reliability, clustering and classi"cation problems. 3. It often occurs as a subproblem in the solution of other problems. For example, minimum spanning tree (MST) algorithms are used in several exact and approximation algorithms for the travelling salesman problem, the multi-terminal #ow problem, the matching problem and the capacitated MST problem. 4. E$cient solution techniques capable of solving large instances exist. The long and rich history of the problem is detailed in the surveys by Pierce [1], Ma$oli [2] and Graham and Hell [3]. Ma$oli's survey takes a broader view and classi"es various forms of optimum undirected tree problems, with emphasis on their computational complexity. The survey of Graham and Hell concentrates solely on the MST; not only surveying the algorithms but also tracing their independent sources. However, this informative survey stops at 1985 emphasising classical algorithms mainly. In the last two decades, e!orts were concentrated on "nding ever faster algorithms, making use of more modern data structures, with each new algorithm bringing the time bound a step closer to linearity. Finally, Karger et al. [4] proposed the only known linear expected-time algorithm for the restricted random access computation model. This is a randomised, recursive algorithm and requires the solution of a related problem, that of verifying whether a given spanning tree is minimum. Numerous authors have studied di!erent forms of the classical algorithms and proposed e$cient implementation techniques (see for example [5]). Reports on computational experiments of various types have also appeared [6,7].
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
769
The continuing interest and developments in the "eld and the discovery of recent algorithms, including the randomised linear-time algorithm of Karger et al. [4], argue for a comprehensive review of the existing methods, classical and modern, and for a comparative empirical study. Most of the modern algorithms are not di$cult to implement. Yet, it is almost standard practice among authors and practitioners to refer to these, but still use one of the classical algorithms which are considerably simpler, though asymptotically slower. The present work aims to verify empirically the conjecture that modern algorithms are also easy to use and have constants of proportionality small enough to make them competitive in practice with older algorithms. In the next two sections, the formal de"nition of the problem and a general solution technique are given. Section 4 is an overview of the classical methods and Section 5 includes a survey of the more recent algorithmic developments. Section 6 empirically evaluates the comparative performance of modern and classical MST algorithms and investigates the feasibility of using the recent methods in solving large problem instances. Section 7 presents a summary and the conclusions.
2. Problem de5nition Given an undirected graph G"(<, E), where < denotes the set of vertices with n"< and E the set of edges with m"E and a real number w(e) for each edge e3E called the weight of edge e, the MSTP is formally de"ned as "nding a spanning tree ¹H on G, such that w(¹H)"min w(e) is 2 CZ2 the minimum taken over all possible spanning trees of G. Given a graph G, a spanning tree ¹ and a cotree edge e"i, j3E!¹, the unique cycle in G consisting of the edge e and the edges of the unique chain in ¹ between i and j is called a fundamental cycle of G relative to ¹ with respect to e. For a graph G and two distinct vertices say i and j, let X be any subset of vertices that contains i but not j and let XM be its complement (i.e., XM "<X), then set X is a cut and the set of edges (i, j) i3X, j3XM , (i, j)3E is called a cutset. The removal of the arcs in a cutset leaves a disconnected subgraph of G. Corresponding to each edge e of a spanning tree ¹ of a connected graph G, there is a unique cutset called the fundamental cutset of ¹ with respect to edge e. The two theorems on the optimality conditions for a MST are stated below without proofs. Two elegant proofs can be found in [8,9]. Theorem 1 (Fundamental cutset optimality). A spanning tree T in a weighted graph is a MST if and only if every edge in the tree is a minimum-weight edge in the fundamental cutset dexned by that edge. Theorem 2 (Fundamental cycle optimality). A spanning tree T in a graph G is a MST if and only if every edge e3E!¹ is a maximum weight edge in the unique fundamental cycle dexned by that edge.
3. General solution The MSTP is solved by a simple incremental, greedy method: the MST is built edge by edge, including appropriate small edges and excluding appropriate large ones. The technique is greedy in the sense that at every stage, the best possible edge is chosen for inclusion in the MST without
770
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
producing a cycle in the subgraph constructed so far or for exclusion from the MST without disconnecting the graph. In his monograph, Tarjan [10] represents the construction process as one of edge colouring. Starting from an initial uncoloured edge set, we colour one edge at a time either blue (included) or red (excluded) according to the following two rules: Blue rule: Select a cutset that does not contain a blue edge. Among the uncoloured edges in the cutset, select one of minimum weight and colour it blue. Red rule: Select a simple cycle containing no red edges. Among the uncoloured edges on the cycle, select one of maximum weight and colour it red. The above blue and red rules are direct applications of the fundamental cutset and fundamental cycle optimality theorems, respectively. Either rule can be applied at any time, making the method non-deterministic. The most important property of the greedy method is that it colours all the edges of any connected graph and maintains a MST containing all of the blue edges and none of the red ones. In the next two sections an overview of the classical methods and a survey of the more recent algorithmic developments are given. Table 1 summarises and presents the existing MST algorithms Table 1 Minimal spanning tree algorithms Algorithm
Special data structures and subproblems used
Time complexity
Boruvka [11] Kruskal (sorted) [12]
O(m log n) O(m(m, n))
Kruskal (not sorted) [12]
Disjoint set union algorithm Disjoint set union (with path compression and merging with rank) Disjoint set union and heapsort
Prim [13]
}
O(n)
Prim Prim Prim
Binary heap d-Heap F-Heap
O(m log n) O(nd log n#m log n) B B O(n log n#m)
Yao [15] Cheriton and Tarjan [16]
Heaps of size k, a selection algorithm Doubly linked queue, leftist heap with lazy meld and lazy deletion, disjoint set union
O(m log log n) O(m log log n)
Fredman and Tarjan [17] Gabow et al. [18]
F-heap, doubly linked queue F-heaps with packets, disjoint set union
O(m(m, n)) O(m log (m, n))
Karger [22] Karger et al. [4]
Randomisation Randomisation, recursion, a linear-time veri"cation algorithm
O(n log n#m) O(m)
(m, n): Inverse Ackermann's function. (m, n)"mini logG n)m/n and log n"n. Expected running time.
O(m log n)
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
771
in "ve categories: classical algorithms (Boruvka, Kruskal, Prim); Prim implementations using di!erent forms of heap structure; algorithms which introduced for the "rst time the special use of heaps and obtained faster running times than the classical algorithms (Yao, Cheriton and Tarjan); algorithms which used F-heaps for the "rst time (Fredman and Tarjan, Gabow et al.); and the more recent algorithms which use randomisation (Karger, Karger et al.). For each algorithm, all the special structures used and the corresponding time complexities are also included.
4. Classical algorithms There are three classical algorithms; namely, Boruvka [10,11], Kruskal [12] and Prim [13]; the last two of which are presented in almost every well-known text in the "eld. All three algorithms use the blue rule. Although algorithms that use the red rule can be designed, until recently, they were thought to be less e$cient. The algorithm of Karger et al. [4] uses the blue rule in conjunction with the red. The blue edges form a forest, consisting of a set of trees that are called blue trees. The most general blue rule algorithm begins with an initial set of blue trees and applies the following colouring step n!1 times: Colouring step: (General) Select a blue tree; "nd a minimum-weight edge incident to this tree and colour it blue. The three classical algorithms di!er in their starting states and colouring steps; in other words, they di!er in the criterion used to select the next edge or edges to be added in each iteration. A brief summary of the three classical algorithms is given below with the conventions of Tarjan [10]. The Boruvka algorithm: Start with the initial set of n blue trees, each consisting of one vertex and no edges, and repeat the following step until there is only one blue tree. Colouring step: For every blue tree ¹, select a minimum weight edge incident to ¹. Colour all selected edges blue. The algorithm builds the trees uniformly throughout the graph. It is, therefore, suitable for use in parallel computations. The Kruskal algorithm: Sort the edges in the order of increasing weights and apply the following step to the edges in the sorted list until the number of blue edges is n!1. Colouring step: If the edge considered has both endpoints in the same blue tree, colour it red; otherwise colour it blue. The algorithm builds up blue trees in an irregular fashion dictated only by edge weights. It is worth noting that the algorithm is best in situations where the edges are given in sorted order or can be sorted fast (like when the weights are small integers, making it possible to employ radix sort) or where the graph is sparse. The Prim algorithm: Use an arbitrary starting vertex s and apply the following step n!1 times. Colouring step: Let ¹ be the blue tree containing s. Select a minimum weight edge incident to ¹ and colour it blue. The algorithm generates only one nontrivial blue tree from a single root. The Boruvka algorithm runs in O(m log n) time (see Table 1). If edges are in disorder with respect to their weight, the Kruskal algorithm requires O(m log n) time. Given a sorted edge list, "nding a MST requires O(m(m, n)) time, assuming the use of the disjoint set union algorithm of Tarjan
772
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
[10], where (m, n) is the inverse Ackermann's function. (m, n) is de"ned as (m, n)"mini*1 A(i, W m/n X )'log n for m*n*1 where Ackermann's function A(i, j) for i, j*1 is given by A(1, j)"2H for j*1, A(i, 1)"A(i!1, 2) for i*2, A(i, j)"A(i!1, A(i, j!1)) for i, j*2. The function A(i, j) grows exponentially fast and hence (m, n) is a very slowly growing function. (m, n))3 for n(2. Hence, for all practical purposes (m, n) is assumed to be a constant not larger than four. The Prim algorithm with an implementation using d-heaps, as proposed by Johnson [14], runs in O(nd log n#m log n). In the case of binary heaps (i.e., d"2), the time complexity is O(m log n). B B Choosing d"U2#m/n V , the time bound becomes O(m log n). If m"(n>C), for some >K L positive , the running time is O(m/). Thus, the algorithm with Johnson's implementation is well suited for dense graphs as well as sparse ones and the method is asymptotically worse than that of Kruskal only if the edges are pre-sorted.
5. Modern algorithms Renewed interest in the MSTP in the last two decades led to faster algorithms. Yao [15] was the "rst to discover an implementation of Boruvka which had a faster asymptotic running time. Cheriton and Tarjan [16] followed by proposing a similar but slightly more practical algorithm. Both algorithms are based on the general blue rule algorithm given above. They both use heaps to implement e$cient Boruvka-like colouring steps and run in O(m log log n) time. With the invention of an e$cient form of heap, called the Fibonacci heap (F-heap), Fredman and Tarjan [17] developed an implementation of the MST algorithm which runs in O(m(m, n)), where (m, n)"mini logGn)m/n with logn"n. In addition, they pointed out that the use of an F-heap in its simplest form in the implementation of the Prim algorithm, i.e., replacing the binary or d-heap with an F-heap, solves the MSTP in O(n log n#m) time. Gabow et al. [18] introduced the use of packets in conjunction with F-heaps and modi"ed the Fredman and Tarjan algorithm to achieve the slightly better time complexity of O(m log (m, n)). The method of Gabow et al. was the fastest theoretically until recently. Currently, the linear expected-time algorithm of Karger et al. [4] stands as theoretically the fastest. It is a randomised and recursive algorithm which also requires the solution of a related problem, that of verifying whether a given spanning tree is minimum. This subproblem itself has been studied earlier, but it was the linear veri"cation algorithm invented by Dixon et al. [19] which made the proof of the existence of a linear expected time MST algorithm possible. The O(m)-time veri"cation algorithm of King [20] seems to be much simpler. In the following, we brie#y describe one example for each of the last three groups in Table 1. The modern algorithms chosen and used in this work are either the fastest or the most practical in each group and are described along with implementation details in [21].
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
773
The Cheriton and Tarjan Algorithm A heap is kept for each blue tree. Each heap holds the edges with at least one endpoint in the tree and which are candidates for becoming blue; the cost of an edge being its key in the heap. The major di!erence from the Boruvka algorithm is that in each iteration we pick one blue tree in the forest for the merge operation. In conjunction with the heap, the selection rule for picking a blue tree for merging plays an important role in beating the O(m log n) time bound. Two alternatives are [16]: E pick uniformly the "rst in a queue of candidate blue trees (implementation with a doubly linked list is su$cient to have an O(1) time operation including deletion from the queue), E pick the smallest candidate tree (less e$cient with an O(n) implementation). This algorithm is asymptotically faster than any of the three classical algorithms for sparse graphs, but is slower by a factor of O(log log n) than the Prim algorithm for dense graphs. The Fredman and Tarjan Algorithm The invention of Fibonacci heaps (F-heaps) by Fredman and Tarjan [17] opened the way to improving some network optimisation algorithms, including those for the shortest path and the MST problems. The most important property of F-heaps is that arbitrary deletion from an n-item heap is possible in O(log n) amortised time, while all other standard heap operations require O(1) amortised time. In many network optimisation algorithms, the number of deletions is relatively small compared to the total number of heap operations. Thus F-heaps provide a very e$cient tool for obtaining asymptotically faster algorithms. The simplest and most straightforward use of an F-heap in the context of the MSTP is in the implementation of the Prim algorithm to "nd the minimum weight edge incident to the blue tree containing the starting vertex. The algorithm requires n deletemin operations and O(m) other heap operations, none of them deletions. Therefore, using F-heaps leads to O(n log n#m) running time (Table 1). The O(m(m, n)) time bound of Fredman and Tarjan is due to a di!erent strategy, which grows a single tree only until its heap of neighbouring vertices exceeds a certain critical size. Then, another tree starting from a new vertex is grown until the heap gets large again. Continuing in this way, the stopping criterion is that every vertex should be in a tree. The algorithm consists of one or more passes of the above form. Each pass implicitly considers the trees of the forest obtained in the previous pass as super vertices. These super vertices are then grown into larger trees in the next pass. After a su$cient number of passes, only the MST remains. Fredman and Tarjan's implementation uses a clean-up operation to achieve condensing the graph before starting each pass. This requires the numbering of the trees consecutively from one and assigning to each vertex the number of the tree containing it. The clean-up requires, a two-pass radix sort for the lexicographic ordering of the edges on the numbers of the trees containing their endpoints, the scanning of the edge list to save the appropriate ones, and the construction for each old tree t of a list of the edges with one endpoint in t after the clean-up. The method of Gabow et al. [18] is an improved version of the Fredman and Tarjan algorithm. The main idea is to group the edges with a common vertex into &packets' and to work only with packet minima. The structure is similar to that of Fredman and Tarjan in general, but additional
774
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
F-heaps are used to handle the packets and to "nd the packet minima. The packets are initialised according to parameter packet size. In order to achieve the claimed time bound O(m log (m, n)), packet size should be chosen as (m, n). Moreover, the heap size selected for the ith pass is chosen as k "2K L and k "2IG\ for i*2. G For practical problems, even if the network is very sparse, packet size is usually small. Hence, the advantage of using packets and using F-heaps for such a small number of elements in each packet is not very big. Therefore, we feel that for most realistic problem sizes, the Fredman and Tarjan algorithm will be comparable to, or even better than, that of Gabow et al. However, this conjecture remains to be tested. The Algorithm of Karger et al. The algorithm of Karger et al. solves the more general problem of "nding a minimum forest in a possibly disconnected graph, but we assume the graph to be connected for simplicity. We also assume that all edge weights are distinct, since an ordering is possible even if they are not. Unlike most algorithms for "nding a MST, the Karger et al. algorithm makes use of both the blue and red rules. The algorithm relies on a random sampling step to discard edges that cannot be in the MST. We need the following terminology to state the result that indicates the e!ectiveness of this edge elimination step. Given an edge (x, y)3E and a forest F in G, F(x, y) denotes the path connecting x and y in F with w (x, y) denoting the maximum weight of an edge on F(x, y). We use the $ convention w (x, y)"R if x and y are not connected in F. An edge is F if w(x, y)'w (x, y) $ $ and F otherwise. It is worth noting that all edges of F are F and no F edges can be in the MST of G as a result of the red rule. There exist O(m)-time algorithms to compute the F edges (for example, [19,20]). Lemma 1 (Karger et al. [4]). Let H be a subgraph obtained from G by including each edge independently with probability p, and let F be the minimum spanning forest of H. The expected number of F edges in G is at most n/p where n is the number of vertices. We omit the proof and only state that the lemma generalises to matroids. There are three major operations, namely the Boruvka step, the random sampling step and the veri"cation step: Boruvka step: Given a graph G, apply the Boruvka algorithm to carry out one colouring step only. Contract the graph G, i.e., replace each blue tree by a single vertex, delete edges whose endpoints are the same after the replacement and delete all but the lowest-weight edge among each set of multiple edges, again after the replacement. Random sampling step: Given a contracted graph G, choose a subgraph H by selecting each edge in G independently with a probability 1/2. Verixcation step: Given any minimum spanning forest F of H-G, "nd all the F edges (both those in H and not in H) and delete them from G to reduce the graph further Each Boruvka step reduces the number of vertices by at least one half. The following algorithm is recursive and generates two subproblems after each execution of step 1.
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
775
Algorithm 1. Given G"(<, E), apply two successive Boruvka steps to the graph to contract G. 2. Apply the random sampling step to the contracted graph to select H. 3. Apply the algorithm recursively producing a minimum spanning forest F of the H formed in step 2. 4. Given F of H, apply the veri"cation step to the subgraph H, which was chosen, and obtain a graph G which is reduced further. 5. Apply the algorithm recursively to G to compute the minimum spanning forest F of G . 6. Return those edges contracted in step 1 together with the edges of F . The worst case and the expected running time of the minimum spanning forest algorithm is O(minn, m log n) and O(m), respectively. It runs in O(m) time with probability 1!exp(!(m)) [4].
6. Computational results Computational testing of the surveyed algorithms was carried out on problem instances of various sizes. All implementations were in Pascal and computations were carried out on an HP735 workstation running under UNIX. To the best knowledge of the authors, this is the "rst report in the literature on a comparative, empirical study of the performance of the algorithms of Boruvka and Karger et al. on problem instances of considerable size. The main objectives of the computational study are to: E Compare the performance of the classical and modern algorithms and investigate empirically the practicability of adopting the recently developed ones among the latter to substitute the former. E Observe the e!ect of varying network sizes and compare the real time performance of the algorithms with the theoretical bounds. E Pinpoint the fastest MST algorithm in practice; at least for the range of problem sizes tested. Random problem instances were generated. The size of the largest problem instances was chosen such that they are solvable on a relatively small computer. The numbers of nodes, ranged, in powers of two, from 512 (0.5K) to 16K. The large resulting networks were chosen to be sparse; with m/n ratios of two, four, eight, sixteen and thirty two. Each n;m combination constitutes a group of "ve di!erent problem instances; all the timings are averages for each group. For each problem instance, nodes were selected randomly from a 1000;1000 grid with a uniform probability density. To ensure connectivity, edges (i, j) j"i#1 were chosen "rst, i.e., each node i was connected to node i#1, then the end nodes of the remaining edges were selected from the complete graph at random. The cost of an edge was calculated as the integer part of the Euclidean distance between its two endpoints. Each problem instance was solved to "nd the corresponding MST using the di!erent algorithms. Dynamic memory management techniques are employed to make handling large networks possible. Since memory management operations are machine and compiler dependent, we consider
776
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
them as general utility operations and do not include their execution times in the following discussion. Results regarding the Kruskal algorithm in relation to sorting methods can be analysed in two ways. One is by keeping n constant and increasing m or vice versa; the other is by increasing the input size while keeping the m/n ratio constant. For the latter, we double both input parameters n and m. It is worth noting that the density of the graph decreases with such an approach. It is well known that none of the existing sorting algorithms is the best all round. Some are good for small m, others for large m; where m is the number of items in the list to be sorted. Insertion sort works well when the list is already partially ordered. Merge sort has the best worst-case behaviour, but it requires more storage than a heap sort. Quick sort has the best average behaviour, but its worst-case behaviour is O(m). In this work, we employed both the quick sort and the heap sort in computing the MST with the Kruskal algorithm. We analyse the results by keeping n constant "rst. Even though it requires some extra memory, quick sort is usually recommended for large m. The results obtained in our tests in sorting rather large lists demonstrate that it is fast and advantageous up to m"16K, but at m"32K, quick sort and heap sort become comparable. Beyond this point, we observe the non-linear character of quick sort and heap sort dominates. Fig. 1 illustrates these thresholds for n"2K. Fig. 2 on the other hand demonstrates the non-linearity of quick sort and the superiority of heap sort for n"8K. Although heap sort is also non-linear, its non-linearity was not observed within the problem range under consideration. The conjecture that heap sort is usually not quite as fast as quick sort does not
Fig. 1. Performance of Kruskal and the sorting algorithms used for n"2K.
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
777
Fig. 2. Performance of Kruskal and the sorting algorithms used for n"8K.
hold for very large m. Once the edges are available in sorted order, the average computing time for Kruskal is extremely small (see Figs. 1 and 2). The second type of analysis, in which the m/n ratio is kept constant while increasing the input size, produced similar results. The gap between quick sort and heap sort increases as the m/n ratio increases for large n. The Prim algorithm performs better with a conventional binary heap than with any other heap. In general, the implementation with d-heaps comes second, but the implementation with F-heaps gains an advantage over it as the number of edges increases (see Table 2). Increasing m/n decreases the di!erence between the F-heap and d-heap implementations, and the implementation with F-heap dominates the one with d-heap for m/n"32, except for small n (n)2K). It is worth noting that implementing the Prim algorithm without the use of heaps is extremely ine$cient comparatively (see column no-hp in Table 2). The results also reveal the quadratic nature of the algorithm. The Boruvka algorithm can be implemented in various ways. In the current study, we used the disjoint set-union algorithm with and without clean-up, i.e., permanently deleting the edges that have both end-points in the same connected component. Figs. 3 and 4 are performance plots. An interesting observation is that the use of clean-up turns out to be counter productive. In our implementation, we "rst form a new edge list in the form of a doubly linked list, so that each time an edge is found to have two end points in the same subtree, it can be deleted permanently from the edge list in O(1) time. However, the overhead of the initial copying process and accessing the edges
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
778
Table 2 Performance of Prim's algorithm using di!erent forms of heaps (s) m
n"0.5K
n"1K
n"2K
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
1K 2K 4K 8K 16K 32K 64K
0.08 0.09 0.09 0.11 0.14 * *
0.01 0.01 0.02 0.03 0.07 * *
0.01 0.01 0.02 0.04 0.06 * *
0.01 0.02 0.03 0.04 0.08 * *
* 0.37 0.38 0.40 0.43 0.48 *
* 0.03 0.03 0.05 0.08 0.15 *
* 0.03 0.03 0.05 0.08 0.13 *
* 0.03 0.04 0.06 0.10 0.16 *
* * 1.88 1.90 1.93 2.00 2.16
* * 0.06 0.08 0.11 0.19 0.34
* * 0.06 0.07 0.11 0.18 0.31
* * 0.06 0.09 0.13 0.21 0.37
m
n"4K
8K 16K 32K 64K 128K 256K
n"8K
n"16K
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
no-hp
d-hp
b-hp
F-hp
12.69 12.73 12.80 12.94 13.25 *
0.14 0.19 0.27 0.44 0.81 *
0.14 0.18 0.26 0.41 0.73 *
0.15 0.21 0.30 0.47 0.81 *
* 54.38 54.46 54.68 55.09 55.75
* 0.33 0.42 0.61 0.99 1.80
* 0.30 0.40 0.57 0.91 1.60
* 0.33 0.46 0.66 1.03 1.75
* * 220.41 220.51 221.86 222.03
* * 0.71 0.92 1.32 2.15
* * 0.66 0.88 1.26 1.98
* * 0.70 0.97 1.40 2.15
no-hp: without any heap; d-hp: with d-heap; b-hp: with binary heap; F-hp: with Fibonacci heap.
in the list is larger than the gain from reducing the number of edges. Although the problems considered are quite large, they require few Boruvka phases and the reduction in the total number of edges does not compensate for the list processing overhead. Nevertheless, in much larger problems, Boruvka's algorithm with clean-up may dominate. Next, we compare one representative of each group of modern algorithms in Table 1. Cheriton and Tarjan, Fredman and Tarjan and Karger et al. are chosen for their practicality or recency as representatives of each group. Figs. 5}8 present their performances, respectively. The Cheriton and Tarjan algorithm is relatively the simplest and the most e$cient in terms of real CPU time for the problem instances considered. This result is veri"ed up to n"16K and m"256K. Due to physical memory limitations, the observed non-linearity beyond n"16K and m"128K was not veri"ed using even larger problem instances. The Fredman and Tarjan algorithm seems to be most interesting from the viewpoint of practical performance. Although it is fast in theory, the break even point seems far away. Moreover, unless n is su$ciently large, it is highly sensitive to the m/n ratio. The sparser the graph, the more ine$cient is the method, which, at least for the range of problem sizes we have used, con#icts with the asymptotic bound. Figs. 6 and 7 are performance plots which illustrate that given n, increasing m/n decreases the running time until the point where the heap size k is larger than n and hence where only one pass is performed. The choice of the parameter k dictates the number of passes. Smaller values of k reduce the time per pass; larger values reduce the number of passes. In other
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
Fig. 3. Performance of Boruvka's algorithm.
Fig. 4. Performance of Boruvka's algorithm w.r.t. to m/n ratio.
779
780
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
Fig. 5. Performance of Cheriton and Tarjan's algorithm.
Fig. 6. Performance of Fredman and Tarjan's algorithm.
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
Fig. 7. Performance of Fredman and Tarjan's algorithm w.r.t. m/n.
Fig. 8. Performance of Karger et al.'s algorithm.
781
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
782
words, the trees that are grown without interruption tend to be small in passes with a small k. Practical results demonstrate that stopping the tree growing process and restarting growing a new tree from scratch very often, following the necessary heap initialisation and other assignments, is ine$cient. Choosing k"2K R, where t is the number of trees before the pass, optimises the running time per pass to be O(m), but this is not observed for inputs which are not su$ciently large. The ine$ciency for small m/n is also partly due to the use of the clean-up process after each pass. Unless n and m are su$ciently large, each additional pass adds a substantial amount of clean-up time. The algorithm of Karger et al. is also not practical for the problem sizes we considered. Due to its recursive nature, we store a reduced network in each subproblem and, hence, spend more time in copying and processing these local networks. Such overheads are either not small or even inordinate. Moreover, it is important to note that the expected time linearity of the algorithm is not observed with the data set used. Table 3 o!ers a comparison of the modern algorithms with the classical for n"8K and n"16K, with m/n ranging between 2 and 32. The results indicate that the conventional binary heap implementation of the Prim algorithm is the fastest for the problem sizes considered. Table 4 indicates roughly the amount of work one has to do to implement each algorithm. The algorithm of Karger et al. is the most di$cult to implement as it also requires the solution of the minimum spanning tree veri"cation and the least-common ancestor subproblems in linear time. A problem related to that addressed in this work; namely, the problem of re-optimisation of solutions, is noteworthy. This problem, also known in the literature as the problem of MST maintenance on dynamic graphs, deals with maintaining a minimum spanning tree on a graph subject to modi"cations (change of edge weights, insertion and deletion of edges and vertices). Neither classical nor modern algorithms provide adequate data structures and mechanisms for fast maintenance. However, there are for this purpose fast methods that use special structures and
Table 3 Performance of MST algorithms (s) n
m
Kruskal
Prim
Boruvka
Cheriton
4K
8K 16K 32K 64K 128K
0.21 0.36 0.66 1.31 2.69
0.14 0.18 0.26 0.41 0.73
0.24 0.44 0.84 1.62 3.39
0.44 0.65 0.96 1.61 2.86
7.53 6.78 0.84 1.44 2.64
1.36 2.19 4.02 7.64 18.77
8K
16K 32K 64K 128K 256K
0.46 0.78 1.44 2.83 5.58
0.30 0.40 0.57 0.91 1.60
0.54 0.99 1.83 3.83 7.42
0.94 1.31 1.99 3.42 5.90
29.91 25.82 1.78 3.06 5.58
2.96 5.17 10.47 26.58
With conventional binary heap. Not performed due to memory limitations.
Fredman
Karger
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
783
Table 4 Code sizes of MST algorithms (bytes) Code
Kruskal
Prim
Boruvka
Cheriton
Fredman
Karger
Source Executable
10664 7168
8788 6656
12456 7680
25761 13312
35742 15872
82121 38656
With conventional binary heap.
techniques (see for example [23]). A brief survey of the MST maintenance problem can be found in [21].
7. Conclusions The paper brie#y surveys the existing algorithms for solving the MSTP and classi"es them according to their similarities or the tools and subproblems they use. The feasibility of using the modern methods, which are more complicated compared to the classical ones, has been investigated. The empirical results lead to the conclusion that while the algorithms developed in recent years are theoretically very important, they require excessive problem sizes to justify their use as an alternative to the classical methods. The problem dimensions we have experimented with are rather large, but still not su$ciently large to make the asymptotically fast algorithms competitive in practice with the older ones. Among modern methods, the Cheriton and Tarjan round robin algorithm is relatively the simplest. It has constants of proportionality small enough to make it the most e$cient and most competitive of the modern methods. This work has helped in observing the e!ect of varying network sizes and in comparing the empirical time performance of the algorithms with the theoretical bounds. In the case of the Fredman and Tarjan algorithm, it is found that extremely large problem sizes are required to reach the break even point and hence the method is not viable to apply in realistic cases. The expected linear-time algorithm of Karger et al. was also found to be not feasible to use, at least for the range of problem sizes we considered. It uses the complicated subproblem of verifying a MST in linear time, which in turn requires the solution of the nearest common ancestor problem in linear time. For the range of problem sizes considered here, the Prim algorithm with a conventional binary heap implementation is found to be the fastest. In view of its simplicity and practicality, it is safe to conclude that the Prim algorithm will continue to serve e$ciently with a conventional binary heap implementation in most cases and possibly with a F-heap implementation in problem instances much larger than those considered in this study.
References [1] Pierce AR. Bibliography on algorithms for shortest path, shortest spanning tree and related circuit routing problems (1956}1974). Networks 1975;5:129}49.
784
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
[2] Ma$oli F. Complexity of optimum undirected tree problems: a survey of recent results. In: Ausiello G, Lucertini M, editors. Analysis and design of algorithms in combinatorial optimization. International Center for Mechanical Sciences. CISM Courses and Lectures, 266. New York: Springer, 1981. p. 107}28. [3] Graham RL, Hell P. On the history of the minimum spanning tree problem. Annals of the History of Computing 1985;7:43}57. [4] Karger DR, Klein PN, Tarjan RE. A randomized linear-time algorithm to "nd minimum spanning trees. Journal of the Association for Computing Machinery 1995;42/2:321}8. [5] Haymond RE, Jarvis JP, Shier DR. Computational methods for minimum spanning tree algorithms. SIAM Journal on Scienti"c and Statistical Computing 1984;5/1:157}74. [6] Glover F, Klingman D, Krishnan R, Padman A. An in-depth empirical investigation of non-greedy approaches for the minimum spanning tree problem. European Journal of Operational Research 1992;56:343}56. [7] Moret BME, Shapiro D. How to "nd a minimum spanning tree in practice. In: Maurer H, editor. New results and new trends in computer science: proceedings, Graz, Austria, June 1991. Lecture Notes in Computer Science, Vol. 555. Berlin: Springer, 1991. p. 192}203. [8] Balakrishnan VK. Network optimization. London: Chapman and Hall, 1995. [9] Gondran M, Minoux M. Graphs and Algorithms. New York: Wiley, 1984. [10] Tarjan RE. Data structures and network algorithms. In CBMS-NFS Regional Conference Series in Applied Mathematics. Philadelphia: Society for Industrial and Applied Mathematics, 1983. p. 44. [11] Boruvka O. O jisteH m probleH mu minimaH lnmH m. PraH ca MoravskeH Pr\ mH rodove\ deckeH Spolec\ nosti 1926;3:37}58 [in Czech], cited in [3,10]. [12] Kruskal JB. On the shortest spanning subtree of a graph and the traveling salesman problem. Proceedings of the American Mathematical Society 1956;7:48}50. [13] Prim RC. Shortest connection networks and some generalizations. Bell System Technical Journal 1957;36:1389}401. [14] Johnson DB. Priority queues with update and "nding minimum spanning trees. Information Processing Letters 1975;4:53}7. [15] Yao A. An O(Elog log <) algorithm for "nding minimum spanning trees. Information Processing Letters 1975;4:21}3. [16] Cheriton D, Tarjan RE. Finding minimum spanning trees. SIAM Journal on Computing 1976;5:724}42. [17] Fredman ML, Tarjan RE. Fibonacci heaps and their uses in improved network optimisation algorithms. Journal of the Association for Computing Machinery 1987;34/3:596}615. [18] Gabow HN, Galil Z, Spencer TH, Tarjan RE. E$cient algorithms for "nding minimum spanning trees in undirected and directed graphs. Combinatorica 1986;6:109}22. [19] Dixon B, Rauch M, Tarjan RE. Veri"cation and sensitivity analysis of minimum spanning trees in linear time. SIAM Journal on Computing 1992;21:1184}92. [20] King V. A simpler minimum spanning tree veri"cation algorithm. In: Akl SG, Dehne F. editors. Algorithms and data structures: Proceedings of the 4th International Workshop, Kingston, Canada. Lecture Notes in Computer Science, 955. Berlin: Springer, 1995. p. 440}8. [21] Bazlamac7 cm CF. Optimised network design: minimum spanning trees and minimum concave-cost problems. Ph.D. thesis, University of Manchester, Institute of Science and Technology, Manchester, England, 1996. [22] Karger DR. Random sampling in matroids, with applications to graph connectivity and minimum spanning trees. In Proceedings of the 34th Annual IEEE Symposium on Foundations of Computer Science. Los Alamitos, California: IEEE Computer Society Press, 1993. pp. 84}93. [23] Eppstein D, Italiano GF, Tamassia R, Tarjan RE, Westbrook J, Yung M. Maintenance of a minimum spanning forest in a dynamic plane graph. Journal of Algorithms 1992;13:33}54.
CuK neyt F. Bazlamac7 c1 is an Assistant Professor in Electrical and Electronics Engineering Department, Middle East Technical University (METU), Ankara, Turkey. He received his B.S. and M.S. degrees, both in electrical and electronics engineering, from the Middle East Technical University and his Ph.D. degree in Computation from the University of Manchester, Institute of Science and Technology (UMIST), Manchester. His current research interests include network
C.F. Bazlamac7 cn, K.S. Hindi / Computers & Operations Research 28 (2001) 767}785
785
and graph algorithms, synthesis and design of computer, communication and transportation networks, applications of graph theory in operations research, metaheuristics and advanced data structures and algorithms. Khalil S. Hindi is a Professor of Systems Engineering at Brunel University, Greater London. His current research interests are in computer-aided management, planning, operation, scheduling and control of engineering systems; particularly manufacturing systems, electric power systems and gas and water distribution and transmission systems, employing computing and mathematical modelling techniques.