- Email: [email protected]
A neural design for solution of the maximal independent set problem
186
European Journal of Operational Research 62 (1992) 186-193 North-Holland
Theory and Methodology
A neural design for solution of the maximal independent set problem Laura I. Burke Department of Industrial Engineering, Lehigh University, Bethlehem, PA 18015, USA Received June 1990; revised January 1991
Abstract: A new, artificial neural structure is presented for generating maximal independent sets of a graph. Although other authors have attempted to solve the maximum independent set problem with analog neural networks, we focus herein on the problem of generating all, or several, maximal independent sets so that the likelihood of 'covering' all nodes in a graph is high. The design proposed is extremely fast, even in simulation on a conventional computer, and does not encounter difficulty determining both maximal and independent sets (as conventional approaches may). Empirical results suggest that the neural approach easily generates several maximal independent sets, and tends to cover all nodes of the graph in the sets generated. In comparison with a heuristic for finding the maximum independent set, the approach also works well; more often than not, it finds larger sets than does the heuristic. Keywords: Neutral networks; maximal independent set; graph theory; constraint satisfaction
Introduction Hopfield and Tank's approach to the traveling salesman problem via an analog neural network representation [5] received attention and stimulated investigation into other combinatorial optimization problems. Typically, however, analysts have attempted to conform to Hopfield and Tank's representation and have further neglected to address operations research issues in their treatment. The maximal independent set problem provides a fertile ground for neural network study, and represents a conundrum which, though treated briefly in previous work [8,9] has had its full neural compatibility overlooked. In this paper, we purposefully choose an instance of the independent set problem that lends itself elegantly to a neural design. Evaluation of the net-
work proceeds on test problem sizes of 50-100 nodes, without restricting test cases to graphs which are only cubic a n d / o r planar.
1.0. Maximal independent set problem The maximal independent (vertex) set problem in graph theory belongs to the class NP-hard in combinatorial optimization. It may be formulated via integer linear programming as a set packing problem. It seeks a vertex set, V,__V
(1)
where V represents the set of all nodes in graph G, independent in the sense that the vertices are pairwise disjoint; i.e, aky=0
for a l l k , j ~ V I
(2)
where A = { a J is the adjacency matrix of the
0377-2217/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved
L.I. Burke / Neuraldesignfor maximal independentset problem undirected graph. Additionally, V1 is maximal in the sense that Vx g Vr,
(3)
for any independent vertex set Vr . A maximal independent set trivially yields a minimal node cover, V~, for the graph: V~ = V - V,
(4)
Additionally, the maximal independent set in G yields a maximal clique in G', the complement graph of G. In many cases, and in the instance of primary interest herein, we must generate not simply one maximal independent set, or even the maximal independent set of maximum cardinality (a maximum independent set), but rather all maximal independent sets in a graph. The reason for wishing to find all (or at least many) can be illustrated by two examples.
1.1. Project selection (Christofides, [2]) Each of n projects in a set {S} = { $ 1 , s 2 . . . . . Sn} has associated resource requirements such that there may arise conflicts among projects if execution is to be simultaneous. Hence, we can graphically represent the set and conflicts as a graph G with nodes corresponding to projects, and an arc (i, j ) between nodes i and j if projects i and j have overlapping resource requirements. An obvious associated objective might be to simultaneously execute as many projects as feasible. If we have a one-time decision to make, the corresponding graph measure we seek is the (or a) maximum (cardinality) independent set. If, however, we have a dynamic system in which we will perform projects at regular intervals, we would prefer not to choose the same projects at each session. The list of all - or a list of several maximal independent sets will serve us best. Thus, at each interval, we seek to execute a feasible set of projects, with the goal of covering as many projects as possible.
1.2. Interceptor scheduling (Ignizio, [6]) Ignizio formulated the defense problem of scheduling interceptors for missiles as a case of conflict scheduling. That is, a missile may be
187
intercepted by interceptors scheduled at different times. These can conflict in a pairwise fashion with other interceptors for the same missile and with interceptors for other missiles. The problem posed by Ignizio lends itself most appropriately to a two-objective formulation: cover as many missiles as possible and schedule as many interceptors as possible. However, as a single objective problem (i.e., fire as many interceptors as possible), it naturally suits a maximal independent set formulation. Moreover, it provides another particularly good example of the need for generation of multiple maximal independent sets. With such a list, a subset of all sets generated may be chosen in order to cover as many missiles as possible.
1.3. Problem for cubic planar graphs Recent approaches to solving the maximum independent set problem focus on restricted graphs; for example, only cubic and planar graphs [1]. Even for such graphs the problem remains NP-complete. However, especially in the case of the two aforementioned applications, such assumptions impose unnecessarily restrictive limits on the problem. We can certainly not guarantee that, in a project selection problem, all projects will conflict with exactly three other projects (satisfying the cubic requirement); nor can we realistically expect that the physical representation of those conflicts will remain planar. The present paper proposes a new, adaptive or artificial neural structure (the next section elaborates) for solution of the maximal independent set problem (and hence the node cover and clique problem as well). We point out a direct analogy between the property of independence in a graph and biological models of cells. We then outline a simple procedure for inducing an artificial neural representation of a graph to yield a series of maximal independent sets. We attempt to determine computational complexity based on a serial implementation of the algorithm. A more important justification of the approach depends on neural compatibility of our algorithm, together with its ability to yield feasible and possibly good solutions. We then appeal to the potential for its ultimate manifestation as a neurally-inspired parallel machine. The transformation discussed herein might stimulate new in-
188
L.L Burke / Neuraldesignfor maximalindependentset problem
sights and ideas about independence problems in graphs and other combinatorial problems, especially in that it does not suffer from many of the shortcomings of the original Hopfield approach to the traveling salesman problem (TSP). Specifically, in contrast to the Hopfield TSP network, our proposed network can guarantee feasibility. The rather general instance of the maximal independent set problem, to generate 'all' such sets, allows for digital rather than analog computation. In turn, the simplicity of the network leads to fast solutions.
starting point. Experimentally, this approach worked well. In the typical Hopfield-type neural network, the state of the network corresponds to a function of the states, or activations, of all nodes. For instance, assume that system state = ~ v i
where u i =- state of node i. Assume further that the state of each individual node is found from a threshold function, Ui
2. R e p r e s e n t a t i o n as a neural structure
The representation of the maximal independent set (MIS) problem on graph G as an artificial neural structure - henceforth referred to as simply a neural structure - is particularly direct. We envision a network sharing properties with both Hopfield networks [5] and Grossberg shunting models [3]. Graph G itself (as described in the previous section) provides the topology of the network, where nodes and connections in the neural system will assume a one-to-one correspondence with nodes and arcs, respectively, in G. Representation of a complete graph naturally requires more storage than a very sparse graph (one with considerably fewer arcs, or, equivalently, with smaller average d ( v ) , where d ( v ) represents the degree of node v). However, for denser graphs, the number of MIS's will be smaller than for sparser graphs by a very direct argument. Thus, for more expensive to represent graphs, computation should be much cheaper. Neurons in the network have activity levels which depend on the input supplied to them via connections. In contrast to neural learning systems for pattern recognition, weights on connections do not adapt. Instead, we fix them at - 1 for all links. Justification for this choice appears in the next section. The system does not actually learn, in any real sense of the word. Rather, it attempts to find stable states. These should correspond to maximal independent sets for our purposes. Once it finds and outputs a stable state (MIS), we need to 'encourage it' to find a new, distinct MIS. Herein, we simply adopt the ad hoc approach of reinitialization followed by a new
(5)
i
=
sgn(wtivj)
(6)
where sgn(x) = (~
ifx>_T,
if x < T,
(7)
and T, a threshold, equals zero for our formulation. The naive system state described in (5) corresponds, in Hopfield terminology, to an energy function of the network. If more nodes are 'on' (have activity 1) then we maximize our system energy. It is more convenient to define the energy function,
E = ½ ( - Evivj)
(8)
i~j
and we can easily show the function to be bounded and monotonic. For the ideal implementation of the algorithm, of course, we envision a hardware system like that described in [5], for example. Conceptually, the hardware consists of processing units and their connections. A weight or strength is associated with these connections, and this is fixed, for our purposes. A processing unit processes only local information; i.e., that which feeds into it from connected units. Units have associated activity levels which are dynamic and dependent on whether or not the weighted activations of connected units (i.e., the information incoming) exceeds a threshold. Such a system satisfies the definition of a neural network, for instance, as stated by HechtNielsen [4]. As so-called neural chips become reality, and a means for implementing neurallycompatible algorithms evolves, the significance of algorithms which are 'neural' becomes self-evi-
L.L Burke / Neural design for maximal independent set problem
dent. Herein we present a neurally-amenable algorithm for generation of maximal independent sets.
3. C o n s t r a i n t s a t i s f a c t i o n - f e a s i b i l i t y
The independence problem and even the maximal independent set problem lends itself perhaps more naturally than any other combinatorial optimization problem to an artificial neural structure implementation. We describe the system's operational ability to ensure feasibility first. Independence constraint (shunting model): By defining a node's state to be 1 (in the independent set) when on and O when off (not in set), and by assigning all weights on all arcs to equal - 1 , we easily achieve the feasibility requirement in (2). To illustrate, consider a distinguished neuron, n. When on, this vertex suppresses the activity of any adjacent neuron by the state update equation in (6). For an arbitrary neuron, j, it is sufficient for only one neuron in its adjacency neighborhood to be activated to suppress the activity of neuron j. Consider the example given in Figure 1. Let the current state o f node activities be given by v ~ = v 2 = 1 , and v 3 = v 4 = 0 . If we assume a sequential update, then the following state changes will result when node activities are updated: v 1 = sgn[(w2,,)(v2) + (w3,1)(v3) = sgn[(-1)(1) + (-1)(0)
+ (w4,1)(u4) ]
+ (-1)(0)]
= O,
v2 = s g n [ ( w i , 2 ) ( V l ) ]
v 3 = sgn[(wl,3)(Vl)
= sgn[O]
+ (w4,3)(u4) ]
189
activity = 1. In a truly parallel update (if both neurons' states could be updated simultaneously), each neuron would inhibit the other. Although we would lose members of our set, we would move toward stability and hence feasibility. Moreover, if we can assume even a very small time difference in the update of the two nodes (what Hopfield termed sequential update), then only one node will lose its activity level and become inactive. Thus, the update equation and negative unit weights ensure that a node having any neighbor on will turn off; if its input is less than zero, it will change to an inactive state. Maximality: The state update equation together with weight definitions also ensure maximality. For instance, for the graph of Figure 1, consider an independent set which we would like to discover to be non-maximal: {2}. Obviously, the independent set {2} is contained in the independent set {2,4}. The system will recognize non-maximal sets by not stabilizing, because the input to node 4, which is zero (because (W3,aXU 3) = ( - - 1X0) = 0) will cause it to become active by (6). In a dual sense to the independence constraint, the weights and update act to ensure maximality by the simple fact that if a node has no neighbor which is on, it will turn on; if its input is zero it will change state to become active. Thus, we can think of an 'on' neuron as being stable if its neighborhood is 'off'; an inactive neuron wants to turn on if it is surrounded by other inactive neurons. Such a system directly parallels Grossberg's 'on-center, off-surround' model [3] for cell activity. In this neurally based model, lateral competition among neurons (a direct result of inhibitory, or negative, weights)
--- sgn[0 + 0]
®
=1, U4 ~---s g n [ ( w 3 , 4 ) ( v 3 ) + ( W l , 4 ) ( V l ) ]
= sgn[( - 1)(1) + 0] =0.
Hence the MIS, VI = {2, 3} will result. The network will not stabilize on anything other than a feasible state, because if an infeasible state exists, the system will perturb itself. Neighboring neurons 1 and 2 cannot both have
\\
Figure 1. Graph with vertex set {1, 2, 3, 4}
190
L.I. Burke / Neural design for maximal independent set problem
results in a 'winning' neuron whose positive activity level suppresses the activities of its immediate neighbors. Such a model usually supposes additional excitatory connections from input patterns. In our case, an even simpler use results. The stable state is naturally achieved by the network. On the other hand, is it possible for the system to oscillate? Or, can there exist a maximal independent set in which a node not belonging to the set is surrounded by similar nodes? The answer is quite simply no, as in such a case the existing set could not be maximal (the node in question could belong to the independent set without upsetting the current set).
4. Complexity and design issues Although computational complexity ought not be directly compared against nonneural algorithms, we undertake such analysis. First we note that each node can clearly compute from local information on|y and thus in a parallel fashion. (We do, however, simulate the results of a sequential update herein.) Initialization of the system at zero activity for all nodes begins each iteration. The search for a maximal independent set begins at an arbitrary node by updating its state. In experiments herein, we update nodes in the same sequence. Thus, for a graph with n nodes, a 'pass' consists of n updates. Once the system stabilizes on a MIS (two consecutive passes are identical), we consider the iteration over. Another iteration can begin by again setting the activity of the system to zero, and beginning update on a different node. Our appro~ich here simply dictated that each iteration begin with a different node; thus, for an n-node graph, n iterations resulted in up to n maximal independent sets. (An obvious extension, which will appear in forthcoming work, allows for additional MIS's via random update of nodes within an iteration.) At worst case, for an iteration, stability will result after n passes. For each update, the number of operations depends on the degree of the node, which is at most n - 1 . Thus, at worst n - 1 multiplication and addition operations, and one comparison, is required. Thus, at each iteration, n * n * 3(n - 1) computations are required, or O(n3). We can perform as many iterations as desired; for instance, as we chose here, we can
perform n iterations, increasing computational complexity to O(n4). When we can take full advantage of parallelism via parallel update, the computational complexity should reduce to some extent. In addition, it should be noted that for a dense graph, the degree of each node is relatively high, thus the multiplication and addition operations approach the worst case figure of n - 1. However, fewer independent sets exist so that fewer perturbations may be performed (armed with this a priori knowledge). On the other hand, for sparse graphs the number of multiplication/addition operations will tend to remain constant and very small (for example, for cubic graphs, d ( v ) = 3 for all nodes). Finally, we note that our experimental system virtually always stabilized after two passes, rather than n. Further, we can restrict the number of passes to achieve stability, which we did by choosing ten passes as the limit. In only one iteration on one 30-node problem did stability require more than two passes (it took three). Smarter implementations should guarantee that the system will indicate its inability to escape a non-stable configuration readily. Moreover, random updates may also contribute to fast generation of distinct sets. Such investigations are in progress, and results are forthcoming.
5. Preliminary experimental results The very apt representation as a neural system naturally finds stable states corresponding to desired sets. For the case where we desire all, or a large number, of MIS, we have no corresponding objective function. If we imposed one, local minima of the energy function would be as acceptable as the one which finds the largest value of E (the largest cardinality MIS). The biggest difficulty encountered is that we wish to guarantee a list of distinct sets. To do so is much more difficult than at first it appears. The guarantee itself corresponds to optimality; thus, once again, we can appeal to the idea of approximation and hope that our approach will tend to generate distinct lists. In the first preliminary test problem, the graph had 25 nodes with 70 arcs (conflicts). Each node had degree ranging from two to four. The graph
L.I. Burke / Neural design for maximal independent set problem
was thus neither cubic nor, as it readily revealed from a sketch, planar. In addition, its adjacency matrix was somewhat sparse, and purposefully so. For sparse graphs, the independence number (size of the largest independent set) is high, whereas for dense graphs it is low [2]. Correspondingly, we can infer that the number of maximal independent sets is larger for a sparse graph than for a dense graph. Thus, the problem posed represents a combinatorially more difficult one than for a dense graph. The network possessed an initial state of zero activity; all V~ = 0. In a sequential update mode, it generated an independent set having cardinality nine which was maximal. By allowing the network to then repeatedly reinitialize at zero and begin update on a different node, it found ten inde-
191
pendent sets, all maximal, nine of which were distinct. The generation of each set usually required only one pass. Testing on 50-100 node problems yielded encouraging results. First, we allowed as many perturbations as nodes, so as many sets as nodes were generated. This ad hoc approach to selecting how to generate hopefully different sets and which to generate produced satisfactory results, although refinements are in progress. Comparisons were made with a n O ( n 3) heuristic [6,7]. Results appear in Table 1. The heuristic, Ignizio's column-sum heuristic [6], operates on the adjacency matrix of the graph G. It attempts to find the maximum independent set, but is only sure to find a maximal independent set. It operates, essentially, by finding the
Table 1 Number of nodes
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
50 50 50 60 60 60 60 60 60 80 80 80 80 80 80 80 80 80 80 80 80 80 80 100 100 100 100 100 100 100 100 100
a Times are in seconds.
Range d(v)
15-28 14-33 4-13 0- 6 34-43 1- 7 0- 4 2- 7 0- 4 2- 9 0- 6 9-24 1- 5 0- 5 14-23 11-23 9-23 11-23 9-23 0- 3 0- 5 20-34 1- 6 0- 5 1- 6 1-10 1-10 0- 4 7-20 4-17 0- 3 5-16
Neural
Column sum heuristic
Avg. time per set a
Set size
Time
0.1174 0.103 0.114 0.025 0.202 0.035 0.025 0.035 0.025 0.046 0.031 0.129 0.024 0.029 0.128 0.129 0.129 0.129 0.119 0.133 0.130 0.20 0.04 0.063 0.028 0.037 0.046 0.023 0.066 0.062 0.022 0.062
4- 8 4- 9 4- 8 31-35 2- 7 13-27 29-33 13-27 29-33 27-35 38-47 10-17 52-55 37-44 10-17 9-15 10-17 11-16 10-17 47-53 39-45 4-16 31-41 100 42-50 32-40 27-33 57-63 17-26 17-29 63-68 17-26
0.01 0.05 0.01 0.16 0.05 0.05 0.10 0.05 0.10 0.10 0.16 0.05 0.21 0.16 0.05 0.10 0.05 0.10 0.05 0.16 0.16 0.05 0.16 0.27 0.16 0.16 0.10 0.21 0.10 0.10 0.21 0.05
Set size 6 7 6 35 6 28 33 28 33 35 46 15 55 44 19 14 19 14 17 53 45 11 43 84 50 38 33 65 25 28 66 26
192
L.I. Burke / Neural design for maximal independent set problem
total adjacency number for each node i, which is just the number of nodes adjacent to node i. It then assigns the node with the largest such number a state value of 1, and uses the adjacency matrix to eliminate all adjacent nodes from further consideration. Using the revised adjacency matrix, the process is repeated. It continues until no more nodes may be assigned state 1. In previous investigations, it proved an efficient, consistent provider of close-to-optimal solutions for large problems [7]. For the neural algorithm, the two-pass to stability rule remained valid for the larger problems. In addition, the neural approach tended to find as large or larger sets (among the many generated) than did the column sum heuristic. Exceptions to this trend appear for problem numbers {6, 8, 15, 17, 23, 28}. Hence, for a total of six out of 32 experiments, the set found by the column sum heuristic was larger than that found by the neural approach (never, however, significantly larger). On the other hand, for a total of 14 out of 32 experiments, the neural approach found larger sets. The remainder of the time the two methods found a largest set having the same size. tn addition, the neural approach also found relatively small independent sets. Contrary to intuition, the minimum maximal independent set is not the null set, and may pose as difficult a problem as the maximum MIS. The neural approach can easily tackle both problems. Typically, the neural approach 'covered' all nodes in its generation of n sets. Finally, empirical results showed that computation times increased at most linearly with problem size. In fact, in some cases the structure of the graph allowed for quicker computation of larger problems. The results reported herein are experimental and preliminary. Assessment continues via larger problems as well as evaluation against both exact and heuristic procedures.
6. Extensions
The maximal independent set problem as stated here represents a constraint satisfaction problem, and its combinatorial nature is enhanced by the requirement of finding (all) distinct maximal independent sets. The neural approach proposed here guarantees efficient generation of
MIS's but cannot guarantee generation of all. Empirically, it tends to find mostly distinct sets. The fact that the constraint of maximality is easily accounted for is important; Christofides [2] points out that the difficulty for most enumerative algorithms is that they generate a large number of independent sets which are found to be non-maximal and hence rejected. Such problems do not afflict the neural approach. As the results in Table 1 suggest, though we cannot guarantee that the approach will generate all MIS's, it appears quite capable of generating several sets for each problem. The size of the sets nearly always varies over a fairly wide range, and since we know each one is maximal (not contained in another), we know that each additional set generated covers additional nodes. Thus the empirical evidence suggests that the approach succeeds in 'covering' a large number of nodes (subject, of course, the constraints of the problem). Moreover, the network is reinitialized at each iteration with a different starting node. Such a strategy encourages the network to generate a MIS with the starting node included. Finding the maximum independent set poses a more complicated problem than of generating several MIS. The difficulty lies in the fact that, by definition, the maximum independent set will not contain as a subset any other MIS. Thus, generating several MIS's will not guarantee finding the maximum MIS. A modified approach suggests itself, however, which extends naturally to the problem of weighted maximum independent sets. First, we can no longer expect digital computation to accomplish our end. An energy function needs to be imposed (see [8]). Longer computation time will result. However, the modifications allow for solution of additional problem instances such as weighted versions of the maximum MIS problem. Since node cover and clique problems can be easily obtained from the solution of the MIS, we will not comment on architectures for their solution at this time (see [8]).
7. Conclusion
The maximal independent set problem naturally lends itself to a neural design. While more
L.I. Burke / Neural design for maximal independent set problem
complicated representations can search for the largest set, and others seek to find the single best answer for a special case of the problem [9], we have found that a useful choice of problem instance best suits a neural design. Moreover, in preliminary tests, the neural design proved capable of generating feasible, mostly distinct solutions for reasonably sized and difficult problem cases. Further testing continues and results will appear in forthcoming articles. The neural approach gives not only large maximal independent sets ('good' solutions to the maximum independent set problem), it addresses the problem of generating multiple such sets so that all nodes, or projects, can be 'assigned'. For the project selection and interceptor scheduling problems, this translates into the desired ability to schedule activities over several time periods so that none conflict and, ultimately, as many projects (interceptors) as possible are executed (fired).
193
References [1] Burns, J., "The maximum independent set problem for cubic planar graphs", Network 19 (1989) 373-378. [2] Christofides, N., Graph Theory: An Algorithmic Approach, Academic Press, New York, 1975. [3] Grossberg, S., "On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems", Biological Cybernetics 21 (1976) 145-159. [4] Hecht-Nielsen, R., Neural Computing, Addison-Wesley, New York, 1990. [5] Hopfield, J., and Tank, D., "Neural computation of decisions in optimization problems", Biological Cybernetics 52 (1985) 141-152. [6] Ignizio, J., "Solving large scheduling problems by minimizing conflict", Simulation March (1978) 75-79. [7] Ignizio, L., "A multicriteria approach to the interceptor scheduling problem", Unpublished Master's Thesis, Pennsylvania State University, 1985. [8] Ramanujam, J., and P., Sadayappan, "Optimization by neural networks", in: Proceedings of the IEEE International Conference on Neural Networks, Vol. H, 1988, 325332. [9] Tagliarini, G., and Page, E., "Solving constraint satisfaction problems with neural networks", in: Proceedings of the IEEE International Conference on Neural Networks, Vol. H, 1988, 741-747.
European Journal of Operational Research 62 (1992) 186-193 North-Holland
Theory and Methodology
A neural design for solution of the maximal independent set problem Laura I. Burke Department of Industrial Engineering, Lehigh University, Bethlehem, PA 18015, USA Received June 1990; revised January 1991
Abstract: A new, artificial neural structure is presented for generating maximal independent sets of a graph. Although other authors have attempted to solve the maximum independent set problem with analog neural networks, we focus herein on the problem of generating all, or several, maximal independent sets so that the likelihood of 'covering' all nodes in a graph is high. The design proposed is extremely fast, even in simulation on a conventional computer, and does not encounter difficulty determining both maximal and independent sets (as conventional approaches may). Empirical results suggest that the neural approach easily generates several maximal independent sets, and tends to cover all nodes of the graph in the sets generated. In comparison with a heuristic for finding the maximum independent set, the approach also works well; more often than not, it finds larger sets than does the heuristic. Keywords: Neutral networks; maximal independent set; graph theory; constraint satisfaction
Introduction Hopfield and Tank's approach to the traveling salesman problem via an analog neural network representation [5] received attention and stimulated investigation into other combinatorial optimization problems. Typically, however, analysts have attempted to conform to Hopfield and Tank's representation and have further neglected to address operations research issues in their treatment. The maximal independent set problem provides a fertile ground for neural network study, and represents a conundrum which, though treated briefly in previous work [8,9] has had its full neural compatibility overlooked. In this paper, we purposefully choose an instance of the independent set problem that lends itself elegantly to a neural design. Evaluation of the net-
work proceeds on test problem sizes of 50-100 nodes, without restricting test cases to graphs which are only cubic a n d / o r planar.
1.0. Maximal independent set problem The maximal independent (vertex) set problem in graph theory belongs to the class NP-hard in combinatorial optimization. It may be formulated via integer linear programming as a set packing problem. It seeks a vertex set, V,__V
(1)
where V represents the set of all nodes in graph G, independent in the sense that the vertices are pairwise disjoint; i.e, aky=0
for a l l k , j ~ V I
(2)
where A = { a J is the adjacency matrix of the
0377-2217/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved
L.I. Burke / Neuraldesignfor maximal independentset problem undirected graph. Additionally, V1 is maximal in the sense that Vx g Vr,
(3)
for any independent vertex set Vr . A maximal independent set trivially yields a minimal node cover, V~, for the graph: V~ = V - V,
(4)
Additionally, the maximal independent set in G yields a maximal clique in G', the complement graph of G. In many cases, and in the instance of primary interest herein, we must generate not simply one maximal independent set, or even the maximal independent set of maximum cardinality (a maximum independent set), but rather all maximal independent sets in a graph. The reason for wishing to find all (or at least many) can be illustrated by two examples.
1.1. Project selection (Christofides, [2]) Each of n projects in a set {S} = { $ 1 , s 2 . . . . . Sn} has associated resource requirements such that there may arise conflicts among projects if execution is to be simultaneous. Hence, we can graphically represent the set and conflicts as a graph G with nodes corresponding to projects, and an arc (i, j ) between nodes i and j if projects i and j have overlapping resource requirements. An obvious associated objective might be to simultaneously execute as many projects as feasible. If we have a one-time decision to make, the corresponding graph measure we seek is the (or a) maximum (cardinality) independent set. If, however, we have a dynamic system in which we will perform projects at regular intervals, we would prefer not to choose the same projects at each session. The list of all - or a list of several maximal independent sets will serve us best. Thus, at each interval, we seek to execute a feasible set of projects, with the goal of covering as many projects as possible.
1.2. Interceptor scheduling (Ignizio, [6]) Ignizio formulated the defense problem of scheduling interceptors for missiles as a case of conflict scheduling. That is, a missile may be
187
intercepted by interceptors scheduled at different times. These can conflict in a pairwise fashion with other interceptors for the same missile and with interceptors for other missiles. The problem posed by Ignizio lends itself most appropriately to a two-objective formulation: cover as many missiles as possible and schedule as many interceptors as possible. However, as a single objective problem (i.e., fire as many interceptors as possible), it naturally suits a maximal independent set formulation. Moreover, it provides another particularly good example of the need for generation of multiple maximal independent sets. With such a list, a subset of all sets generated may be chosen in order to cover as many missiles as possible.
1.3. Problem for cubic planar graphs Recent approaches to solving the maximum independent set problem focus on restricted graphs; for example, only cubic and planar graphs [1]. Even for such graphs the problem remains NP-complete. However, especially in the case of the two aforementioned applications, such assumptions impose unnecessarily restrictive limits on the problem. We can certainly not guarantee that, in a project selection problem, all projects will conflict with exactly three other projects (satisfying the cubic requirement); nor can we realistically expect that the physical representation of those conflicts will remain planar. The present paper proposes a new, adaptive or artificial neural structure (the next section elaborates) for solution of the maximal independent set problem (and hence the node cover and clique problem as well). We point out a direct analogy between the property of independence in a graph and biological models of cells. We then outline a simple procedure for inducing an artificial neural representation of a graph to yield a series of maximal independent sets. We attempt to determine computational complexity based on a serial implementation of the algorithm. A more important justification of the approach depends on neural compatibility of our algorithm, together with its ability to yield feasible and possibly good solutions. We then appeal to the potential for its ultimate manifestation as a neurally-inspired parallel machine. The transformation discussed herein might stimulate new in-
188
L.L Burke / Neuraldesignfor maximalindependentset problem
sights and ideas about independence problems in graphs and other combinatorial problems, especially in that it does not suffer from many of the shortcomings of the original Hopfield approach to the traveling salesman problem (TSP). Specifically, in contrast to the Hopfield TSP network, our proposed network can guarantee feasibility. The rather general instance of the maximal independent set problem, to generate 'all' such sets, allows for digital rather than analog computation. In turn, the simplicity of the network leads to fast solutions.
starting point. Experimentally, this approach worked well. In the typical Hopfield-type neural network, the state of the network corresponds to a function of the states, or activations, of all nodes. For instance, assume that system state = ~ v i
where u i =- state of node i. Assume further that the state of each individual node is found from a threshold function, Ui
2. R e p r e s e n t a t i o n as a neural structure
The representation of the maximal independent set (MIS) problem on graph G as an artificial neural structure - henceforth referred to as simply a neural structure - is particularly direct. We envision a network sharing properties with both Hopfield networks [5] and Grossberg shunting models [3]. Graph G itself (as described in the previous section) provides the topology of the network, where nodes and connections in the neural system will assume a one-to-one correspondence with nodes and arcs, respectively, in G. Representation of a complete graph naturally requires more storage than a very sparse graph (one with considerably fewer arcs, or, equivalently, with smaller average d ( v ) , where d ( v ) represents the degree of node v). However, for denser graphs, the number of MIS's will be smaller than for sparser graphs by a very direct argument. Thus, for more expensive to represent graphs, computation should be much cheaper. Neurons in the network have activity levels which depend on the input supplied to them via connections. In contrast to neural learning systems for pattern recognition, weights on connections do not adapt. Instead, we fix them at - 1 for all links. Justification for this choice appears in the next section. The system does not actually learn, in any real sense of the word. Rather, it attempts to find stable states. These should correspond to maximal independent sets for our purposes. Once it finds and outputs a stable state (MIS), we need to 'encourage it' to find a new, distinct MIS. Herein, we simply adopt the ad hoc approach of reinitialization followed by a new
(5)
i
=
sgn(wtivj)
(6)
where sgn(x) = (~
ifx>_T,
if x < T,
(7)
and T, a threshold, equals zero for our formulation. The naive system state described in (5) corresponds, in Hopfield terminology, to an energy function of the network. If more nodes are 'on' (have activity 1) then we maximize our system energy. It is more convenient to define the energy function,
E = ½ ( - Evivj)
(8)
i~j
and we can easily show the function to be bounded and monotonic. For the ideal implementation of the algorithm, of course, we envision a hardware system like that described in [5], for example. Conceptually, the hardware consists of processing units and their connections. A weight or strength is associated with these connections, and this is fixed, for our purposes. A processing unit processes only local information; i.e., that which feeds into it from connected units. Units have associated activity levels which are dynamic and dependent on whether or not the weighted activations of connected units (i.e., the information incoming) exceeds a threshold. Such a system satisfies the definition of a neural network, for instance, as stated by HechtNielsen [4]. As so-called neural chips become reality, and a means for implementing neurallycompatible algorithms evolves, the significance of algorithms which are 'neural' becomes self-evi-
L.L Burke / Neural design for maximal independent set problem
dent. Herein we present a neurally-amenable algorithm for generation of maximal independent sets.
3. C o n s t r a i n t s a t i s f a c t i o n - f e a s i b i l i t y
The independence problem and even the maximal independent set problem lends itself perhaps more naturally than any other combinatorial optimization problem to an artificial neural structure implementation. We describe the system's operational ability to ensure feasibility first. Independence constraint (shunting model): By defining a node's state to be 1 (in the independent set) when on and O when off (not in set), and by assigning all weights on all arcs to equal - 1 , we easily achieve the feasibility requirement in (2). To illustrate, consider a distinguished neuron, n. When on, this vertex suppresses the activity of any adjacent neuron by the state update equation in (6). For an arbitrary neuron, j, it is sufficient for only one neuron in its adjacency neighborhood to be activated to suppress the activity of neuron j. Consider the example given in Figure 1. Let the current state o f node activities be given by v ~ = v 2 = 1 , and v 3 = v 4 = 0 . If we assume a sequential update, then the following state changes will result when node activities are updated: v 1 = sgn[(w2,,)(v2) + (w3,1)(v3) = sgn[(-1)(1) + (-1)(0)
+ (w4,1)(u4) ]
+ (-1)(0)]
= O,
v2 = s g n [ ( w i , 2 ) ( V l ) ]
v 3 = sgn[(wl,3)(Vl)
= sgn[O]
+ (w4,3)(u4) ]
189
activity = 1. In a truly parallel update (if both neurons' states could be updated simultaneously), each neuron would inhibit the other. Although we would lose members of our set, we would move toward stability and hence feasibility. Moreover, if we can assume even a very small time difference in the update of the two nodes (what Hopfield termed sequential update), then only one node will lose its activity level and become inactive. Thus, the update equation and negative unit weights ensure that a node having any neighbor on will turn off; if its input is less than zero, it will change to an inactive state. Maximality: The state update equation together with weight definitions also ensure maximality. For instance, for the graph of Figure 1, consider an independent set which we would like to discover to be non-maximal: {2}. Obviously, the independent set {2} is contained in the independent set {2,4}. The system will recognize non-maximal sets by not stabilizing, because the input to node 4, which is zero (because (W3,aXU 3) = ( - - 1X0) = 0) will cause it to become active by (6). In a dual sense to the independence constraint, the weights and update act to ensure maximality by the simple fact that if a node has no neighbor which is on, it will turn on; if its input is zero it will change state to become active. Thus, we can think of an 'on' neuron as being stable if its neighborhood is 'off'; an inactive neuron wants to turn on if it is surrounded by other inactive neurons. Such a system directly parallels Grossberg's 'on-center, off-surround' model [3] for cell activity. In this neurally based model, lateral competition among neurons (a direct result of inhibitory, or negative, weights)
--- sgn[0 + 0]
®
=1, U4 ~---s g n [ ( w 3 , 4 ) ( v 3 ) + ( W l , 4 ) ( V l ) ]
= sgn[( - 1)(1) + 0] =0.
Hence the MIS, VI = {2, 3} will result. The network will not stabilize on anything other than a feasible state, because if an infeasible state exists, the system will perturb itself. Neighboring neurons 1 and 2 cannot both have
\\
Figure 1. Graph with vertex set {1, 2, 3, 4}
190
L.I. Burke / Neural design for maximal independent set problem
results in a 'winning' neuron whose positive activity level suppresses the activities of its immediate neighbors. Such a model usually supposes additional excitatory connections from input patterns. In our case, an even simpler use results. The stable state is naturally achieved by the network. On the other hand, is it possible for the system to oscillate? Or, can there exist a maximal independent set in which a node not belonging to the set is surrounded by similar nodes? The answer is quite simply no, as in such a case the existing set could not be maximal (the node in question could belong to the independent set without upsetting the current set).
4. Complexity and design issues Although computational complexity ought not be directly compared against nonneural algorithms, we undertake such analysis. First we note that each node can clearly compute from local information on|y and thus in a parallel fashion. (We do, however, simulate the results of a sequential update herein.) Initialization of the system at zero activity for all nodes begins each iteration. The search for a maximal independent set begins at an arbitrary node by updating its state. In experiments herein, we update nodes in the same sequence. Thus, for a graph with n nodes, a 'pass' consists of n updates. Once the system stabilizes on a MIS (two consecutive passes are identical), we consider the iteration over. Another iteration can begin by again setting the activity of the system to zero, and beginning update on a different node. Our appro~ich here simply dictated that each iteration begin with a different node; thus, for an n-node graph, n iterations resulted in up to n maximal independent sets. (An obvious extension, which will appear in forthcoming work, allows for additional MIS's via random update of nodes within an iteration.) At worst case, for an iteration, stability will result after n passes. For each update, the number of operations depends on the degree of the node, which is at most n - 1 . Thus, at worst n - 1 multiplication and addition operations, and one comparison, is required. Thus, at each iteration, n * n * 3(n - 1) computations are required, or O(n3). We can perform as many iterations as desired; for instance, as we chose here, we can
perform n iterations, increasing computational complexity to O(n4). When we can take full advantage of parallelism via parallel update, the computational complexity should reduce to some extent. In addition, it should be noted that for a dense graph, the degree of each node is relatively high, thus the multiplication and addition operations approach the worst case figure of n - 1. However, fewer independent sets exist so that fewer perturbations may be performed (armed with this a priori knowledge). On the other hand, for sparse graphs the number of multiplication/addition operations will tend to remain constant and very small (for example, for cubic graphs, d ( v ) = 3 for all nodes). Finally, we note that our experimental system virtually always stabilized after two passes, rather than n. Further, we can restrict the number of passes to achieve stability, which we did by choosing ten passes as the limit. In only one iteration on one 30-node problem did stability require more than two passes (it took three). Smarter implementations should guarantee that the system will indicate its inability to escape a non-stable configuration readily. Moreover, random updates may also contribute to fast generation of distinct sets. Such investigations are in progress, and results are forthcoming.
5. Preliminary experimental results The very apt representation as a neural system naturally finds stable states corresponding to desired sets. For the case where we desire all, or a large number, of MIS, we have no corresponding objective function. If we imposed one, local minima of the energy function would be as acceptable as the one which finds the largest value of E (the largest cardinality MIS). The biggest difficulty encountered is that we wish to guarantee a list of distinct sets. To do so is much more difficult than at first it appears. The guarantee itself corresponds to optimality; thus, once again, we can appeal to the idea of approximation and hope that our approach will tend to generate distinct lists. In the first preliminary test problem, the graph had 25 nodes with 70 arcs (conflicts). Each node had degree ranging from two to four. The graph
L.I. Burke / Neural design for maximal independent set problem
was thus neither cubic nor, as it readily revealed from a sketch, planar. In addition, its adjacency matrix was somewhat sparse, and purposefully so. For sparse graphs, the independence number (size of the largest independent set) is high, whereas for dense graphs it is low [2]. Correspondingly, we can infer that the number of maximal independent sets is larger for a sparse graph than for a dense graph. Thus, the problem posed represents a combinatorially more difficult one than for a dense graph. The network possessed an initial state of zero activity; all V~ = 0. In a sequential update mode, it generated an independent set having cardinality nine which was maximal. By allowing the network to then repeatedly reinitialize at zero and begin update on a different node, it found ten inde-
191
pendent sets, all maximal, nine of which were distinct. The generation of each set usually required only one pass. Testing on 50-100 node problems yielded encouraging results. First, we allowed as many perturbations as nodes, so as many sets as nodes were generated. This ad hoc approach to selecting how to generate hopefully different sets and which to generate produced satisfactory results, although refinements are in progress. Comparisons were made with a n O ( n 3) heuristic [6,7]. Results appear in Table 1. The heuristic, Ignizio's column-sum heuristic [6], operates on the adjacency matrix of the graph G. It attempts to find the maximum independent set, but is only sure to find a maximal independent set. It operates, essentially, by finding the
Table 1 Number of nodes
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
50 50 50 60 60 60 60 60 60 80 80 80 80 80 80 80 80 80 80 80 80 80 80 100 100 100 100 100 100 100 100 100
a Times are in seconds.
Range d(v)
15-28 14-33 4-13 0- 6 34-43 1- 7 0- 4 2- 7 0- 4 2- 9 0- 6 9-24 1- 5 0- 5 14-23 11-23 9-23 11-23 9-23 0- 3 0- 5 20-34 1- 6 0- 5 1- 6 1-10 1-10 0- 4 7-20 4-17 0- 3 5-16
Neural
Column sum heuristic
Avg. time per set a
Set size
Time
0.1174 0.103 0.114 0.025 0.202 0.035 0.025 0.035 0.025 0.046 0.031 0.129 0.024 0.029 0.128 0.129 0.129 0.129 0.119 0.133 0.130 0.20 0.04 0.063 0.028 0.037 0.046 0.023 0.066 0.062 0.022 0.062
4- 8 4- 9 4- 8 31-35 2- 7 13-27 29-33 13-27 29-33 27-35 38-47 10-17 52-55 37-44 10-17 9-15 10-17 11-16 10-17 47-53 39-45 4-16 31-41 100 42-50 32-40 27-33 57-63 17-26 17-29 63-68 17-26
0.01 0.05 0.01 0.16 0.05 0.05 0.10 0.05 0.10 0.10 0.16 0.05 0.21 0.16 0.05 0.10 0.05 0.10 0.05 0.16 0.16 0.05 0.16 0.27 0.16 0.16 0.10 0.21 0.10 0.10 0.21 0.05
Set size 6 7 6 35 6 28 33 28 33 35 46 15 55 44 19 14 19 14 17 53 45 11 43 84 50 38 33 65 25 28 66 26
192
L.I. Burke / Neural design for maximal independent set problem
total adjacency number for each node i, which is just the number of nodes adjacent to node i. It then assigns the node with the largest such number a state value of 1, and uses the adjacency matrix to eliminate all adjacent nodes from further consideration. Using the revised adjacency matrix, the process is repeated. It continues until no more nodes may be assigned state 1. In previous investigations, it proved an efficient, consistent provider of close-to-optimal solutions for large problems [7]. For the neural algorithm, the two-pass to stability rule remained valid for the larger problems. In addition, the neural approach tended to find as large or larger sets (among the many generated) than did the column sum heuristic. Exceptions to this trend appear for problem numbers {6, 8, 15, 17, 23, 28}. Hence, for a total of six out of 32 experiments, the set found by the column sum heuristic was larger than that found by the neural approach (never, however, significantly larger). On the other hand, for a total of 14 out of 32 experiments, the neural approach found larger sets. The remainder of the time the two methods found a largest set having the same size. tn addition, the neural approach also found relatively small independent sets. Contrary to intuition, the minimum maximal independent set is not the null set, and may pose as difficult a problem as the maximum MIS. The neural approach can easily tackle both problems. Typically, the neural approach 'covered' all nodes in its generation of n sets. Finally, empirical results showed that computation times increased at most linearly with problem size. In fact, in some cases the structure of the graph allowed for quicker computation of larger problems. The results reported herein are experimental and preliminary. Assessment continues via larger problems as well as evaluation against both exact and heuristic procedures.
6. Extensions
The maximal independent set problem as stated here represents a constraint satisfaction problem, and its combinatorial nature is enhanced by the requirement of finding (all) distinct maximal independent sets. The neural approach proposed here guarantees efficient generation of
MIS's but cannot guarantee generation of all. Empirically, it tends to find mostly distinct sets. The fact that the constraint of maximality is easily accounted for is important; Christofides [2] points out that the difficulty for most enumerative algorithms is that they generate a large number of independent sets which are found to be non-maximal and hence rejected. Such problems do not afflict the neural approach. As the results in Table 1 suggest, though we cannot guarantee that the approach will generate all MIS's, it appears quite capable of generating several sets for each problem. The size of the sets nearly always varies over a fairly wide range, and since we know each one is maximal (not contained in another), we know that each additional set generated covers additional nodes. Thus the empirical evidence suggests that the approach succeeds in 'covering' a large number of nodes (subject, of course, the constraints of the problem). Moreover, the network is reinitialized at each iteration with a different starting node. Such a strategy encourages the network to generate a MIS with the starting node included. Finding the maximum independent set poses a more complicated problem than of generating several MIS. The difficulty lies in the fact that, by definition, the maximum independent set will not contain as a subset any other MIS. Thus, generating several MIS's will not guarantee finding the maximum MIS. A modified approach suggests itself, however, which extends naturally to the problem of weighted maximum independent sets. First, we can no longer expect digital computation to accomplish our end. An energy function needs to be imposed (see [8]). Longer computation time will result. However, the modifications allow for solution of additional problem instances such as weighted versions of the maximum MIS problem. Since node cover and clique problems can be easily obtained from the solution of the MIS, we will not comment on architectures for their solution at this time (see [8]).
7. Conclusion
The maximal independent set problem naturally lends itself to a neural design. While more
L.I. Burke / Neural design for maximal independent set problem
complicated representations can search for the largest set, and others seek to find the single best answer for a special case of the problem [9], we have found that a useful choice of problem instance best suits a neural design. Moreover, in preliminary tests, the neural design proved capable of generating feasible, mostly distinct solutions for reasonably sized and difficult problem cases. Further testing continues and results will appear in forthcoming articles. The neural approach gives not only large maximal independent sets ('good' solutions to the maximum independent set problem), it addresses the problem of generating multiple such sets so that all nodes, or projects, can be 'assigned'. For the project selection and interceptor scheduling problems, this translates into the desired ability to schedule activities over several time periods so that none conflict and, ultimately, as many projects (interceptors) as possible are executed (fired).
193
References [1] Burns, J., "The maximum independent set problem for cubic planar graphs", Network 19 (1989) 373-378. [2] Christofides, N., Graph Theory: An Algorithmic Approach, Academic Press, New York, 1975. [3] Grossberg, S., "On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems", Biological Cybernetics 21 (1976) 145-159. [4] Hecht-Nielsen, R., Neural Computing, Addison-Wesley, New York, 1990. [5] Hopfield, J., and Tank, D., "Neural computation of decisions in optimization problems", Biological Cybernetics 52 (1985) 141-152. [6] Ignizio, J., "Solving large scheduling problems by minimizing conflict", Simulation March (1978) 75-79. [7] Ignizio, L., "A multicriteria approach to the interceptor scheduling problem", Unpublished Master's Thesis, Pennsylvania State University, 1985. [8] Ramanujam, J., and P., Sadayappan, "Optimization by neural networks", in: Proceedings of the IEEE International Conference on Neural Networks, Vol. H, 1988, 325332. [9] Tagliarini, G., and Page, E., "Solving constraint satisfaction problems with neural networks", in: Proceedings of the IEEE International Conference on Neural Networks, Vol. H, 1988, 741-747.