- Email: [email protected]
Graph multidimensional scaling with self-organizing maps
Information Sciences 143 (2002) 159–180 www.elsevier.com/locate/ins
Graph multidimensional scaling with self-organizing maps Eric Bonabeau Icosystem Corporation, 545 Concord Avenue, Cambridge, MA 02138, USA Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA Received 10 March 2000; received in revised form 10 September 2000; accepted 31 October 2001
Abstract Self-organizing maps (SOM) are unsupervised, competitive neural networks used to project high-dimensional data onto a low-dimensional space. In this paper it is shown that SOM can be used to perform multidimensional scaling (MDS) on graphs. The SOM-based approach is applied to two families of random graphs and three real-world networks. Ó 2002 Elsevier Science Inc. All rights reserved. Keywords: Self-organizing maps; Multidimensional scaling; Graphs
1. Introduction 1.1. Graph multidimensional scaling Complex networks of relationships, that arise in numerous disciplines, from psychology to economics, experimental physics or social sciences, can often be represented by graphs [1,21,27]. Such graphs can be weighted, possibly with negative weights, directed or undirected, etc. It is usually convenient to be able to visualize graphs in the form of diagrams, preferably embedded in a twodimensional space, where interpretation by a human observer is facilitated. Automated procedures for drawing graphs are extremely useful, especially for
E-mail address: [email protected] (E. Bonabeau). 0020-0255/02/$ - see front matter Ó 2002 by Elsevier Science Inc. All rights reserved. PII: S 0 0 2 0 - 0 2 5 5 ( 0 2 ) 0 0 1 9 1 - 3
160
E. Bonabeau / Information Sciences 143 (2002) 159–180
large graphs. Many graph drawing algorithms have been designed in the past three decades [1,29,30]. Multidimensional scaling (MDS) is a general method that is used to visualize metric or non-metric high-dimensional data [10], and is especially useful when data is available in the form of pairwise dissimilarity values between data points. The method described in this paper is an implicit MDS algorithm for graphs, based on unsupervised neural networks called selforganizing maps (SOM) [18,24,25]. In their usual form, SOM allow the projection of high-dimensional data onto low-dimensional spaces, while preserving characteristic features of the data. It was previously shown [3] that SOM provide a natural framework for graph drawing, if the local topology is variable instead of fixed, that is, if nodes in the network have a variable number of neighbors. The SOM-based approach is a variant of the Quinn– Breur force-directed placement algorithm in VLSI [28], which has been applied to graph layout by several authors [2,5,11,12,14,15,22]. Although SOM do not have the flexibility of other graph drawing algorithms [9,11] (the layout style cannot be easily selected), they are easy to implement, can be used to lay out weighted graphs, graphs with negative connection weights, and directed graphs. It will be shown in this paper that representing a graph with the SOM-based approach amounts to applying a form of MDS to the graph, where the pairwise dissimilarity between nodes of the graph is the L1 distance between nodes in the space defined by the graph’s adjacency matrix. This type of approach has been used for VLSI placement [7,17,33], but had never been studied as a general heuristic for arbitrary graph representation and graph MDS. Several authors have formulated graph drawing in terms of an optimization problem, in which some energy function has to be minimized (e.g. [11,15,22]). The energy function, which integrates features that will make a drawing appealing and/or useful, is the equivalent of the stress function of MDS [10], although, to the best of my knowledge, none of these authors has made an explicit connection between graph drawing and MDS. Reasons for making this connection explicit are twofold. Firstly, the field of graph drawing can only benefit from the huge body of work in MDS [10]. Secondly, because MDS and clustering are close relatives, thinking of graph drawing as MDS can help design clustering and partitioning algorithms for graphs. Indeed, Kernighan and Lin [23], in their seminal paper on graph partitioning, noted that MDS was good at identifying clusters in a graph prior to partitioning (a simple clustering algorithm, such as k-means [20], can then be applied to the spatial representation of the graph). As another example, the SOM-based approach can be considered a graph partitioning algorithm [4] as well as a graph drawing algorithm [3]. Kuntz et al. [26] discuss similar issues and deal with them using a different, agent-based, algorithm.
E. Bonabeau / Information Sciences 143 (2002) 159–180
161
1.2. Outline of the paper SOM for graphs are described in Section 2. Results of the application of SOM to two families of random graphs and three real-world networks are presented in Section 3.
2. Self-organizing maps 2.1. Two dimensions Let G ¼ ðV ; EÞ be a non-directed graph. V ¼ fvi gi¼1;:::;n is the set of n vertices and E, a subset of V V , the set of edges, of cardinal jEj. E can be represented as a connectivity or adjacency matrix ½eij of edge weights, where eij 6¼ 0 if ðvi ; vj Þ 2 E and eij ¼ 0 if ðvi ; vj Þ 62 E. eij is the weight of edge ðvi ; vj Þðeii ¼ 1 by conventionÞ. The first-order neigborhood of vertex vi is defined by ð1Þ N 1 ðvi Þ ¼ vj 2 V n N 0 ðvi Þeij 6¼ 0 ; where, by convention, N 0 ðvi Þ ¼ fvi g. The cth order neighborhood of vi is defined by recurrence by c 1 [ c d c 1 N ðvi Þ9vl 2 N ðvi Þ; e1j 6¼ 0 : ð2Þ N ð vi Þ ¼ v j 2 V d¼0
2
Let wi 2 R be the positional vector of vertex vi , and xm 2 R2 be the mth training vector, where training vectors are drawn according to a uniform spatial distribution over, for example, a square area of linear size 1. At each time step, a training vector is presented. A vertex viðmÞ is selected. iðmÞ is defined by iðmÞ ¼ arg minvj 2V kxm wj k, where k k is the L1 norm in the plane. Let T and m P 1 be the total simulation time and current time, respectively, expressed in number of training vectors. The coordinates of viðmÞ and of all vertices bef ðmÞ longing to [c¼1 N c ðviðmÞ Þ, where f ðmÞ is an integer function of m, are updated wiðmÞ ðm þ 1Þ wiðmÞ ðmÞ þ aðmÞ xm wiðmÞ ðmÞ ; ð3Þ
1 wj ðmÞ þ c eiðmÞj bðmÞ wiðmÞ ðm þ 1Þ wj ðmÞ ð4Þ wj ðm þ 1Þ for vj 2 N c ðviðmÞ Þ; c 6 f ðmÞ: aðmÞ and bðmÞ are two decaying functions: aðmÞ ¼ a1 T =ðgm þ T Þ and bðmÞ ¼ b1 T =ðgm þ T Þ, where g, a1 and b1 are tunable parameters. aðmÞ and bðmÞ play a role equivalent to that of a fast cooling schedule in simulated annealing. f ðmÞ is used to fine tune the range of influence of the winning node, which should be large initially and progressively decrease. In the examples given below, we use f ðmÞ 6¼ 1 only for geometric graphs, where it is necessary because connections are by definition local in the plane, and therefore the range
162
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 1. (a) Initial distribution of vertices on the portion of plane ½0; 1 ½0; 1 , for a random graph Gð25; 4; 0:8; 0:01Þ. (b) The algorithm is able to find ‘‘natural’’clusters in this graph. Training vectors are presented according to a uniform distribution over ½0; 1 ½0; 1 . Vertices distribute themselves in space in such a way that exactly c ¼ 4 clusters of vertices appear, that correspond to the four clusters of the graph. T ¼ 2000, f ðmÞ ¼ 1. For m 6 T =3: aðmÞ ¼ a1 T =ðgm þ T Þ and bðmÞ ¼ b1 T =ðgm þ T Þ, g ¼ 50, a1 ¼ 0:6, b1 ¼ 0:4. For m > T =3, bðmÞ ¼ expð kwj wiðmÞ k=dc Þ for vj 2 N 1 ðviðmÞ Þ, dc ¼ 0:17.
E. Bonabeau / Information Sciences 143 (2002) 159–180
163
of influence of a node is too localized if f ðmÞ ¼ 1; otherwise, this procedure is computationally costly, and it appears f ðmÞ ¼ 1 is sufficient. Notice than in Eq. (4) vertex positions are updated so as to get closer to wiðmÞ and not to xm as is usual in SOM [18,24,25]. The eiðmÞj term in Eq. (4) allows to treat the case or arbitrary connection weights: for example, if eiðmÞj > 0, wj gets closer to wiðmÞ , while if eiðmÞj < 0, wj is moved further away from wiðmÞ . 2.2. Three dimensions The method described in Section 2.1 can easily be generalized for representing graphs in three dimensions: assuming that wi 2 R3 and xm 2 R3 , and that k k is the L1 norm in R3 , the application of the same algorithm as in Section 2.2, including Eqs. (3) and (4) which are still valid, leads to a threedimensional representation of the graph. This can be especially useful to increase readability and MDS accuracy when the natural structures of the graph are deployed in more than two dimensions. In two dimensions, some nodes may be ‘‘false’’ neighbors, that is, neighbors in the plane but not in attribute space: increasing dimension by one allows to get rid of some of the false neighbors.
3. Results 3.1. Random graphs with clusters The algorithm was tested on random graphs Gðn; c; pi ; pe Þ [16], where n is the number of vertices per cluster, c is the number of clusters, pi is the probability that two vertices within a cluster are connected, and pe is the probability that two vertices belonging to different clusters are connected. Let us set for simplicity eij ¼ 1 if ðvi ; vj Þ 2 E and eij ¼ 0 if ðvi ; vj Þ 62 E. Training vectors xm are presented according to a uniform distribution over the portion of plane considered ð½0; 1 ½0; 1 Þ. Initially, all vertices are randomly placed in this area. In order to avoid some vertices oscillating back and forth on the plane because they are subject to conflicting constraints, bðmÞ is modified when m > T =3 : bðmÞ ¼ expð kwj wiðmÞ k=dc Þ for vj 2 N 1 ðviðmÞ Þ, where dc is a characteristic distance beyond which it is pointless to drag nodes. Fig. 1 illustrates the fact that the algorithm is able to find ‘‘natural’’ clusters in a graph Gð25; 4; 0:8; 0:01Þ: Fig. 1(a) shows the initial random distribution of nodes in ½0; 1 ½0; 1 ; Fig. 1(b) shows that, after T ¼ 2000 training vectors, vertices distribute themselves in space in such a way that exactly c ¼ 4 clusters of vertices appear, that correspond to the four clusters of the graph. Note that the application of a clustering algorithm to the vertex positions obtained after
164
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 2. (a) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 1(a). k k is the L1 norm in the plane, and dð Þ is the L1 norm for graphs. (b) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 1(b).
E. Bonabeau / Information Sciences 143 (2002) 159–180
165
T ¼ 2000 training vectors would yield an optimal partitioning of the graph: the SOM-based algorithm is indeed a very efficient approach to partitioning [4]. The algorithm can now be evaluated as a mapping from graph space onto the plane, that is, as a MDS algorithm: we expect nodes which are far apart (respectively close) in the graph to lie far away from (respectively close to) each other on the plane, and vice versa. Let us introduce the L1 distance dðvi ; vj Þ between two nodes in a graph: n X d vi ; vj ¼ n 1 eik ejk :
ð5Þ
k¼1
In the present case, where eij ¼ 0 or 1, we have dðvi ; vj Þ ¼ n 1 jDðqðvi Þ; qðvj ÞÞj, where qðvi Þ ¼ fvj 2 V ; eij 6¼ 0g is the set of edges adjacent to vi , including vi , D is the symmetric difference ðDðA; BÞ ¼ ðA [ BÞ ðA \ BÞ), and j j denotes the number of elements of a set. The use of D expresses the fact that vertices that possess many common adjacent edges are close in the graph, whereas those with only a few or no adjacent edges in common are far apart in the graph. In a random graph of the type studied here, we expect to have two characteristic distances in the graph: the distance between vertices that belong to the same cluster, and the distance between vertices that do not belong to the same cluster. Fig. 1(a) shows kwi wj k as a function of dðvi ; vj Þ for the same graph and the same distributions of vertex positions in the plane as in Fig. 1(a): the two expected clouds of points are observed, and there is, of course, no significant correlation between kwi wj k and dðvi ; vj Þ. Fig. 2(b) shows the same relationship after T ¼ 2000 training vectors (vertex positions are those of Fig. 1(b): the distance between the positions of vertices in the plane representation is strongly positively correlated to their distance in the graph (P < 0:005), with two clouds of points corresponding to the two characteristic distances in the graph. As the reader can easily convince himself, it is not possible to build a general 2d isometric embedding of a graph with more than three nodes; nevertheless the algorithm is able to generate a good mapping of the graph onto the plane. The choice of the L1 norm in the design of the SOM-based algorithm rather than another norm, such as the Euclidian norm, was dictated by the finding that MDS using a Euclidian norm does not lead to a reliable detection of a graph’s clusters [13]. Hubert and Arabie [19] studied MDS algorithms with the L1 norm, concluded that gradient-based techniques were inadequate and proposed a more sophisticated technique which, because of its complexity, was applied to relatively small data sets. The SOM-based method, although it is an implicit MDS method, can be applied to large data sets. Fig. 3 shows the representation of a graph Gð25; 4; 0:8; 0:01Þ in three dimensions: here, wi 2 ½0; 1 ½0; 1 ½0; 1 and training vectors xm are presented according to a uniform distribution over ½0; 1 ½0; 1 ½0; 1 . Again, the four clusters can be clearly visualized, although not better than in two dimensions
166
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 3. Representation of a graph Gð25; 4; 0:8; 0:01Þ in three dimensions. wi 2 ½0; 1 ½0; 1 ½0; 1 and training vectors xm are presented according to a uniform distribution over ½0; 1 ½0; 1 ½0; 1 . Vertices distribute themselves in space in such a way that exactly c ¼ 4 clusters of vertices appear, that correspond to the four clusters of the graph, but visualization is not better than in two dimensions (Fig. 1(b)). Same parameter values as in Fig. 1(b).
(Fig. 1(b)). Fig. 4 shows kwi wj k as a function of dðvi ; vj Þ for the graph representation of Fig. 3, where k k is the L1 norm in R3 : Fig. 4 is similar to Fig. 2(b). 3.2. Random geometric graphs Fig. 5(c) shows the representation of a randomly generated geometric graph with 200 vertices, shown on Fig. 5(a), found by the algorithm. In geometric graphs, which are used in the context of numerical modeling and finite element simulation, vertices, characterized by Euclidean coordinates, are located on a plane and are locally connected to other vertices that lie within a radius of connectivity R (here, R ¼ 0:18). Vertices are initially randomly distributed in the unit square (Fig. 5(b)). We set f ðmÞ ¼ 3 for 1 6 m 6 T ¼ 2000 to increase the range of influence of nodes. The unfolding of the graph to approximate the training vector distribution leads to a remarkable representation where vertices are located on the plane in such a way that connections are local. Fig. 6(a) shows kwi wj k as a function of dðvi ; vj Þ for the same graph and the same distribution of vertex positions in the plane as in Fig. 5(a): one cloud of points is observed and there is no correlation between kwi wj k and
E. Bonabeau / Information Sciences 143 (2002) 159–180
167
Fig. 4. kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 3. k k is the L1 norm in R3 and dð Þ is the L1 norm for graphs.
dðvi ; vj Þ. Fig. 6(b) shows the same relationship after T ¼ 2000 training vectors (vertex positions are those of Fig. 5(b)): the distance between the positions of vertices in the plane representation is strongly positively correlated to their distance in the graph. Here again, the algorithm has done a good job at performing a form of MDS: for a specialist of geometric graphs, this is obvious upon visual inspection of Fig. 5(c). 3.3. Topological organization of the primate cortical visual system Young [31] studied the parcellation and connections of the primate cortical visual system and gave an ‘‘objective’’ representation of its topological organization using MDS. In his 2d visualization, points representing areas that have very similar afferent and efferent cortico-cortical connections are close together, whereas points representing areas that have very different patterns of connections are far apart. The entries of the connectivity matrix
168
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 5. (a) Geometric graph of 200 vertices in a square area of linear size 1, where the radius of connectivity is R ¼ 0:18 (L1 norm in the plane). (b) Initial distribution of vertices. (c) Representation of the graph found by the algorithm at T ¼ 2000. Same parameters as Fig. 1(b) except that f ðmÞ ¼ 3.
he used take on three values: eij ¼ 2 for reciprocal connections between areas i and j, eij ¼ 1 for one-way connections, and eij ¼ 0 for connections that have been tested for and found absent or connections which are not known. As a test of success of MDS, he used the entries of the connectivity matrix as a measure of the ‘‘closeness’’ of two areas: the aim of MDS was then to obtain a representation such that distances between points of the representation are as close as possible to the reverse rank order of the ‘‘proximities’’ between areas in the matrix. The problem with this is the small number of distance values (three values) in the connectivity matrix. Using the L1 norm for graphs, which is consistent with the problem at hand,
E. Bonabeau / Information Sciences 143 (2002) 159–180
169
Fig. 6. (a) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 5(b). (b) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 5(c).
170
E. Bonabeau / Information Sciences 143 (2002) 159–180
one can obtain a better validation of the approach, although the resulting structure is remarkably similar to the one obtained by Young [31]. Fig. 7 shows the 2d representation obtained after application of the SOM-based approach. As reported by Young [31], several organizational features of the connectional topology are apparent in Fig. 7: the two dimensions of the structure correspond roughly to the anterior–posterior (left to right in Fig. 7) and to the dorsal–ventral (bottom to top) spatial distribution of the areas in the brain. A more detailed description of the structure can be found in [31]. Fig. 8(a) shows kwi wj k as a function of eij , and a significant negative correlation is observed (P < 0:001), as expected. This relationship was the measure of success used by Young [31]. Despite the significance of the negative correlation, it is clear that there are too few values of eij for the interpretation to be unquestionable. Fig. 8(b) shows kwi wj k as a function of dðvi ; vj Þ for the same graph representation: here again, there is a significant correlation (though now positive, of course) (P < 0:001), but this time the interpretation is made easier by the fact that d can take on many more values. Fig. 9 shows a representation of the same graph when the distribution of training vectors is no longer uniform over ½0; 1 ½0; 1 but uniform over
Fig. 7. Representation of a graph given by a connectivity matrix which can be found in [31]. Same parameters as in Fig. 1(b). Node labels (cortical areas): 1: V1, 2: V2, 3: V3, 4: Vp, 5: V3a, 6: V4, 7: VOT, 8: V4t, 9: MT, 10: MSTd, 11: MSTI, 12: FST, 13: PITd, 14: PITv, 15: CITd, 16: CITv, 17: AITd, 18: AITv, 19: STPp, 20: STPa, 21: TF, 22: TH, 23: PO, 24: PIP, 25: LIP, 26: VIP, 27: DP, 28: A7a, 29: FEF, 30: A46.
E. Bonabeau / Information Sciences 143 (2002) 159–180
171
Fig. 8. (a) kwi wj k vs eij for the graph representation of Fig. 7. (b) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 7.
172
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 9. Representation of the same graph as the one represented in Fig. 7. Same parameters as in Fig. 1(b) except that training vectors xm are uniformly distributed over ð½0; 0:4 ½0; 1 Þ[ ð½0:6; 0:1 ½0; 1 Þ.
Fig. 10. Representation in three dimensions of the same graph as the one represented in Fig. 7. wi 2 ½0; 1 ½0; 1 ½0; 1 and training vectors xm are presented according to a uniform distribution over ½0; 1 ½0; 1 ½0; 1 . Same parameter values as in Fig. 1(b).
E. Bonabeau / Information Sciences 143 (2002) 159–180
173
½0; 0:4 ½0:6; 1 , which leads to an implicit bi-partitioned representation of the graph with a ‘‘no-node’s land’’ in the 0:4 < x < 0:6 band. Application of this biased distribution of training vectors leads to a segregation of areas according to the anterior–posterior axis. Finally, Fig. 10 shows a 3d representation of the same graph. Although node labels are not represented for the sake of clarity, it appears clearly that the structure is quasi-2d, with only about 20% nodes located out of the planar component. It is likely that this near-planar (somewhat circular) structure reflects a system that is divided into two gross streams, which possess a common origin in the occipital cortex and reconverge after somewhat segregated processing [32]. Fig. 11 shows kwi wj k as a function of dðvi ; vj Þ for the graph representation of Fig. 10, where k k is the L1 norm in R3 : Fig. 11 is similar to Fig. 8(b), certainly because the structure possesses a strong 2d component, so that a 3d representation does not add a significant amount of information.
Fig. 11. kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 10. k k is the L1 norm in R3 and dð Þ is the L1 norm for graphs.
174
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 12. Representations of a work network in a technical laboratory. Algorithm parameters are identical to those of Fig. 1(b). (a) Only ties that correspond to daily interactions are taken into account. (b) Ties that correspond to daily and weekly interactions are taken into account. (c) Ties that correspond to daily, weekly and monthly interactions are taken into account. (d) Ties that correspond to daily, weekly, monthly and half-yearly interactions are taken into account. (e) Ties that correspond to daily, weekly, monthly, half-yearly and yearly interactions are taken into account. Data used by permission of Valdis Krebs (www.orgnet.com).
In addition to these results, it is worth remembering that the SOM-based approach to MDS is a force-directed algorithm: vertices exert forces to drag their neighbors with them as they move. It is not absurd to think that the genesis of the cortical topological organization may rely on a similar mechanism: neurons move toward certain areas while attracting neurons they are connected to through physical forces, axons being real physical objects. It this were to be the case, the algorithm would not only allow us to analyze the
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 12. (continued).
175
176
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 12. (continued).
connectivity structure, it would also replicate the very formation of the structure! 3.4. Network of workers in a technical laboratory I have applied the MDS technique to the visualization of a network of work relationships in a technical laboratory. The network involves 55 people. The bidirectional ties were determined through a survey and sorted according to the frequency of the interactions between the workers (daily, weekly, monthly, half-yearly and yearly). Figs. 12(a)–(d) show representations of the network when certain ties only are taken into account: ties that correspond to daily interactions for Fig. 12(a), daily and weekly for Fig. 12(b), daily to monthly for Fig. 12(c), and daily to half-yearly for Fig. 12(d). Several ‘‘hubs’’ appear in Fig. 12(a): employees 36, 52 and 55 are central in this network, especially 55. When weekly interactions are taken into account, one, surprisingly, observes a ‘‘spontaneous’’ partitioning of the network into two sub-networks and nodes 27, 42, 48 and 55 are located between the two subnetworks (Fig. 12(b)). While node 55 still seems to be central, nodes 27 and 42 have replaced nodes 36 and 52. When monthly interactions are taken into account, the same partitioning phenomenon is apparent (Fig. 12(c)), with node 19 lying in the middle and a set of other nodes (3, 18, 23, 25, 32, 39, 43, 45 and perhaps 53) located at the
E. Bonabeau / Information Sciences 143 (2002) 159–180
177
periphery of the structure: the nodes at the periphery lie there because they are weakly connected to the rest of the network. When half-yearly interactions are taken into account, the partition is no longer apparent (Fig. 12(d)), but there still are nodes that lie at the periphery, most of them identified in Fig. 12(c), plus nodes 8, 16 and 28. Including yearly interactions does not significantly modify the picture obtained in Fig. 12(d). Interpreting these results is of course not easy, but applying the SOM-based algorithm clearly allowed us to identify key properties of the network. Since the networks represented in Figs. 12(a) and (b) are certainly the most ‘‘relevant’’ to the understanding of the day-today work dynamics, the representations tell us that some nodes are particularly central, and that there may exist two ‘‘natural’’ (and yet unplanned) groups of people, with many intra-group interactions and significantly fewer inter-group interactions. 3.5. Food webs Finally, the approach has been applied to particular ecological networks called community food webs, which include all the species or all the kinds of
Fig. 13. Representation of the California tidal flat community food web [6]. The algorithm parameters are identical to those of Fig. 1(b). The entry matrix is not symmetric. Node labels (species or groups of organisms): 1—primary producers, 2—detritus, 3— protozoa, 4—crustacea, 5— Callianassa, 6—shore crabs, 7—Phoronopsis, 8—Urechis, 9—clams, 10—Lumbrinereis, Notomastus, 11—heron, 12—Clevelandia, 13—wading brids, 14—stingray, 15—Nereis, Glycera, 16—nemerteans, 17—terns, shorebirds, 18—cabezon, 19—flounder, 20—Cerebratulus, 21—loon, cormorant, 22—striped bass, 23—scoter, 24—gulls, 25—man.
178
E. Bonabeau / Information Sciences 143 (2002) 159–180
organisms (there may be several classes of organisms within a species—stage of life, size, etc.—or several taxonomically related species may be represented by the same node in the web) found in a particular habitat, and trophic relationships between them, that is, ‘‘who eats whom’’ [6,8]. Food webs are obviously directed graphs, which suggests that connections should not be treated as symmetric. Here only mother-vertices (with outgoing edges) drag daughter-vertices (with incoming edges) when they move: vertices with outgoing edges are predator species and vertices with incoming edges are species predated upon. In other words predator species drag prey species. Predator species that are not predated upon are expected to lie at the periphery of the unit square. Fig. 13 shows a representation of the California tidal flat food web, the adjacency matrix of which can be found in [6]. The matrix, which has entries for 25 species including man, is binary: eij ¼ 1 if species j is a predator of species i, and eij ¼ 0 otherwise. Fig. 14 shows kwi wj k as a function of eij , and a significant negative correlation is observed (P < 0:001). The representation reveals, as expected, major predator species: the corners of the unit square are
Fig. 14. kwi wj k vs eij for the graph representation of Fig. 13.
E. Bonabeau / Information Sciences 143 (2002) 159–180
179
occupied by man (#25), birds (#11,#13 and #21). Keystone species can also be seen: they are species located at the center of the representation with many connections (examples are #2 and #6). In summary the modified algorithm has been able to highlight the structure of the food web. Treating connections as symmetric would not allow to reveal the predator–prey structure but would still clearly allow to observe the keystone species.
4. Conclusion The SOM-based approach to graph drawing amounts to performing a form of MDS on graphs which may also explain why it is so efficient at partitioning [4]. Future work will be aimed at making the connection between MDS and SOM-based graph drawing more rigorous and at testing different kinds of distance functions other than the L1 distance used in the present paper. This approach is very easy to implement and works particularly well on large graphs because a large graph can more easily ‘‘learn’’ the distribution of training vectors: it can therefore be used in many situations for exploratory network data analysis.
Acknowledgements My work is supported by the Interval Research, Inc. Fellowship in Adaptive Computation at the Santa Fe Institute. I would like to thank Valdis Krebs for insightful discussions and for providing me with the technical lab data.
References [1] G.D. Battista, P. Eades, R. Tamassia, I.G. Tollis, Algorithms for drawing graphs: an annotated bibliography, Comput. Geom. 4 (1994) 235–282. [2] B. Becker, G. Hotz, On the optimal layout of planar graphs with fixed boundary, SIAM J. Comput. 16 (1987) 946–972. [3] E. Bonabeau, F. Henaux, Self-organizing maps for drawing large graphs, Inf. Process. Lett. 67 (1998) 177–184. [4] E. Bonabeau, F. Henaux, Monte Carlo partitioning of graphs using self-organizing maps, Int. J. Mod. Phys. C 9 (1998) 1107–1119. [5] F.J. Brandenburg, M. Himsolt, C. Rohrer, An experimental comparison of force-directed and randomized graph drawing algorithms, in: Proceedings of the Graph Drawing ’95, Lecture Notes in Computer Science, 1027, 1995, pp. 76–87. [6] F. Briand, Environmental control of food web structure, Ecology 64 (1983) 253–263. [7] R.-I. Chang, P.-Y. Hsiao, VLSI circuit placement with rectilinear modules using three-layer force-directed self-organizing maps, IEEE Trans. Neural Networks 8 (1997) 1049–1064. [8] J.E. Cohen, F. Briand, C.M. Newman, Community Food Webs, Springer, Berlin, 1990.
180
E. Bonabeau / Information Sciences 143 (2002) 159–180
[9] M.K. Coleman, D.S. Parker, Aesthetics-based graph layout for human consumption, Software – Pract. Exper. 26 (1996) 1–25. [10] T.F. Cox, M.A.A. Cox, Multidimensional Scaling, Chapman & Hall, London, 1994. [11] R. Davidson, D. Harel, Drawing graphs nicely using simulated annealing, ACM Trans. Graphics 15 (1996) 301–331. [12] P. Eades, A heuristic for graph drawing, Congr. Numer. 42 (1984) 149–160. [13] H. de Frayssex, P. Kuntz, Pagination of large-scale networks; embedding a graph in Rn for effective partitioning, Algorithms Rev. 2 (1992) 105–112. [14] A. Frick, A. Ludwig, H. Mehldau, A fast adaptive layout algorithm for undirected graphs, in: Proceedings of the Graph Drawing ’94, Lecture Notes in Computer Science, 894, 1994, pp. 388–403. [15] T.M.J. Fruchterman, E.M. Reingold, Graph drawing by force-directed placement, Software – Pract. Exper. 21 (1991) 1129–1164. [16] J. Garbers, H.J. Pr€ omel, A. Steger, Finding clusters in VLSI circuits, in: A. SangiovanniVincentinelli, S. Goto (Eds.), Proceedings of the IEEE International Conference on ComputerAided Design, IEEE Computer Society Press, Silver spring, MD, 1990, pp. 520–523. [17] A. Hemani, A. Postula, Cell placement by self-organization, Neural Networks 3 (1990) 377–383. [18] J. Hertz, A. Krogh, R.G. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley, Redwood City, CA, 1991. [19] L. Hubert, P. Arabie, Multidimensional scaling in the city-block metric; a combinatorial approach, J. Classification 9 (1992) 211–236. [20] A.K. Jain, R.C. Dubes, Algorithms for Clustering Data, Prentice-Hall, Englewood Cliffs, NJ, 1988. [21] T. Kamada, Visualizing Abstract Objects and Relations, World Scientific, Singapore, 1989. [22] T. Kamada, S. Kawai, An algorithm for drawing general undirected graphs, Inf. Process. Lett. 31 (1989) 7–15. [23] B. Kernighan, S. Lin, An efficient heuristic procedure for partitioning graphs, Bell Syst. Tech. J. 49 (1970) 291–307. [24] T. Kohonen, Self-organization and associative memory, Springer, Berlin, 1989. [25] T. Kohonen, The self-organizing map, Proc. IEEE 78 (1990) 1464–1480. [26] P. Kuntz, D. Snyers, P. Layzell, A stochastic heuristic for visualising graph clusters in a bidimensional space prior to partitioning, J. of Heuristics 5 (1999) 327–351. [27] T. Lengauer, Combinatorial algorithms for integrated circuit layout, Wiley, Stuttgart, 1990. [28] N. Quinn, M. Breur, A force directed component placement procedure for printed circuit boards, IEEE Trans. Circuits and Systems CAS-26 (1979) 377–388. [29] R. Tamassia, G. Battista, C. Battini, Automatic graph drawing and readability of diagrams, IEEE Trans. Syst. Man Cybern. 18 (1988) 61–79. [30] W.T. Tutte, How to draw a graph, Proc. London Math. Soc. 13 (1963) 743–768. [31] M.P. Young, Objective analysis of the topological organization of the primate cortical visual system, Nature 358 (1992) 152–155. [32] M.P. Young, Scaling and brain connectivity, Nature 369 (1994) 449–450. [33] C.-X. Zhang, D.A. Mlynski, Mapping and hierarchical self-organizing neural networks for VLSI placement, IEEE Trans. Neural Networks 8 (1997) 299–314.
Graph multidimensional scaling with self-organizing maps Eric Bonabeau Icosystem Corporation, 545 Concord Avenue, Cambridge, MA 02138, USA Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA Received 10 March 2000; received in revised form 10 September 2000; accepted 31 October 2001
Abstract Self-organizing maps (SOM) are unsupervised, competitive neural networks used to project high-dimensional data onto a low-dimensional space. In this paper it is shown that SOM can be used to perform multidimensional scaling (MDS) on graphs. The SOM-based approach is applied to two families of random graphs and three real-world networks. Ó 2002 Elsevier Science Inc. All rights reserved. Keywords: Self-organizing maps; Multidimensional scaling; Graphs
1. Introduction 1.1. Graph multidimensional scaling Complex networks of relationships, that arise in numerous disciplines, from psychology to economics, experimental physics or social sciences, can often be represented by graphs [1,21,27]. Such graphs can be weighted, possibly with negative weights, directed or undirected, etc. It is usually convenient to be able to visualize graphs in the form of diagrams, preferably embedded in a twodimensional space, where interpretation by a human observer is facilitated. Automated procedures for drawing graphs are extremely useful, especially for
E-mail address: [email protected] (E. Bonabeau). 0020-0255/02/$ - see front matter Ó 2002 by Elsevier Science Inc. All rights reserved. PII: S 0 0 2 0 - 0 2 5 5 ( 0 2 ) 0 0 1 9 1 - 3
160
E. Bonabeau / Information Sciences 143 (2002) 159–180
large graphs. Many graph drawing algorithms have been designed in the past three decades [1,29,30]. Multidimensional scaling (MDS) is a general method that is used to visualize metric or non-metric high-dimensional data [10], and is especially useful when data is available in the form of pairwise dissimilarity values between data points. The method described in this paper is an implicit MDS algorithm for graphs, based on unsupervised neural networks called selforganizing maps (SOM) [18,24,25]. In their usual form, SOM allow the projection of high-dimensional data onto low-dimensional spaces, while preserving characteristic features of the data. It was previously shown [3] that SOM provide a natural framework for graph drawing, if the local topology is variable instead of fixed, that is, if nodes in the network have a variable number of neighbors. The SOM-based approach is a variant of the Quinn– Breur force-directed placement algorithm in VLSI [28], which has been applied to graph layout by several authors [2,5,11,12,14,15,22]. Although SOM do not have the flexibility of other graph drawing algorithms [9,11] (the layout style cannot be easily selected), they are easy to implement, can be used to lay out weighted graphs, graphs with negative connection weights, and directed graphs. It will be shown in this paper that representing a graph with the SOM-based approach amounts to applying a form of MDS to the graph, where the pairwise dissimilarity between nodes of the graph is the L1 distance between nodes in the space defined by the graph’s adjacency matrix. This type of approach has been used for VLSI placement [7,17,33], but had never been studied as a general heuristic for arbitrary graph representation and graph MDS. Several authors have formulated graph drawing in terms of an optimization problem, in which some energy function has to be minimized (e.g. [11,15,22]). The energy function, which integrates features that will make a drawing appealing and/or useful, is the equivalent of the stress function of MDS [10], although, to the best of my knowledge, none of these authors has made an explicit connection between graph drawing and MDS. Reasons for making this connection explicit are twofold. Firstly, the field of graph drawing can only benefit from the huge body of work in MDS [10]. Secondly, because MDS and clustering are close relatives, thinking of graph drawing as MDS can help design clustering and partitioning algorithms for graphs. Indeed, Kernighan and Lin [23], in their seminal paper on graph partitioning, noted that MDS was good at identifying clusters in a graph prior to partitioning (a simple clustering algorithm, such as k-means [20], can then be applied to the spatial representation of the graph). As another example, the SOM-based approach can be considered a graph partitioning algorithm [4] as well as a graph drawing algorithm [3]. Kuntz et al. [26] discuss similar issues and deal with them using a different, agent-based, algorithm.
E. Bonabeau / Information Sciences 143 (2002) 159–180
161
1.2. Outline of the paper SOM for graphs are described in Section 2. Results of the application of SOM to two families of random graphs and three real-world networks are presented in Section 3.
2. Self-organizing maps 2.1. Two dimensions Let G ¼ ðV ; EÞ be a non-directed graph. V ¼ fvi gi¼1;:::;n is the set of n vertices and E, a subset of V V , the set of edges, of cardinal jEj. E can be represented as a connectivity or adjacency matrix ½eij of edge weights, where eij 6¼ 0 if ðvi ; vj Þ 2 E and eij ¼ 0 if ðvi ; vj Þ 62 E. eij is the weight of edge ðvi ; vj Þðeii ¼ 1 by conventionÞ. The first-order neigborhood of vertex vi is defined by ð1Þ N 1 ðvi Þ ¼ vj 2 V n N 0 ðvi Þeij 6¼ 0 ; where, by convention, N 0 ðvi Þ ¼ fvi g. The cth order neighborhood of vi is defined by recurrence by c 1 [ c d c 1 N ðvi Þ9vl 2 N ðvi Þ; e1j 6¼ 0 : ð2Þ N ð vi Þ ¼ v j 2 V d¼0
2
Let wi 2 R be the positional vector of vertex vi , and xm 2 R2 be the mth training vector, where training vectors are drawn according to a uniform spatial distribution over, for example, a square area of linear size 1. At each time step, a training vector is presented. A vertex viðmÞ is selected. iðmÞ is defined by iðmÞ ¼ arg minvj 2V kxm wj k, where k k is the L1 norm in the plane. Let T and m P 1 be the total simulation time and current time, respectively, expressed in number of training vectors. The coordinates of viðmÞ and of all vertices bef ðmÞ longing to [c¼1 N c ðviðmÞ Þ, where f ðmÞ is an integer function of m, are updated wiðmÞ ðm þ 1Þ wiðmÞ ðmÞ þ aðmÞ xm wiðmÞ ðmÞ ; ð3Þ
1 wj ðmÞ þ c eiðmÞj bðmÞ wiðmÞ ðm þ 1Þ wj ðmÞ ð4Þ wj ðm þ 1Þ for vj 2 N c ðviðmÞ Þ; c 6 f ðmÞ: aðmÞ and bðmÞ are two decaying functions: aðmÞ ¼ a1 T =ðgm þ T Þ and bðmÞ ¼ b1 T =ðgm þ T Þ, where g, a1 and b1 are tunable parameters. aðmÞ and bðmÞ play a role equivalent to that of a fast cooling schedule in simulated annealing. f ðmÞ is used to fine tune the range of influence of the winning node, which should be large initially and progressively decrease. In the examples given below, we use f ðmÞ 6¼ 1 only for geometric graphs, where it is necessary because connections are by definition local in the plane, and therefore the range
162
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 1. (a) Initial distribution of vertices on the portion of plane ½0; 1 ½0; 1 , for a random graph Gð25; 4; 0:8; 0:01Þ. (b) The algorithm is able to find ‘‘natural’’clusters in this graph. Training vectors are presented according to a uniform distribution over ½0; 1 ½0; 1 . Vertices distribute themselves in space in such a way that exactly c ¼ 4 clusters of vertices appear, that correspond to the four clusters of the graph. T ¼ 2000, f ðmÞ ¼ 1. For m 6 T =3: aðmÞ ¼ a1 T =ðgm þ T Þ and bðmÞ ¼ b1 T =ðgm þ T Þ, g ¼ 50, a1 ¼ 0:6, b1 ¼ 0:4. For m > T =3, bðmÞ ¼ expð kwj wiðmÞ k=dc Þ for vj 2 N 1 ðviðmÞ Þ, dc ¼ 0:17.
E. Bonabeau / Information Sciences 143 (2002) 159–180
163
of influence of a node is too localized if f ðmÞ ¼ 1; otherwise, this procedure is computationally costly, and it appears f ðmÞ ¼ 1 is sufficient. Notice than in Eq. (4) vertex positions are updated so as to get closer to wiðmÞ and not to xm as is usual in SOM [18,24,25]. The eiðmÞj term in Eq. (4) allows to treat the case or arbitrary connection weights: for example, if eiðmÞj > 0, wj gets closer to wiðmÞ , while if eiðmÞj < 0, wj is moved further away from wiðmÞ . 2.2. Three dimensions The method described in Section 2.1 can easily be generalized for representing graphs in three dimensions: assuming that wi 2 R3 and xm 2 R3 , and that k k is the L1 norm in R3 , the application of the same algorithm as in Section 2.2, including Eqs. (3) and (4) which are still valid, leads to a threedimensional representation of the graph. This can be especially useful to increase readability and MDS accuracy when the natural structures of the graph are deployed in more than two dimensions. In two dimensions, some nodes may be ‘‘false’’ neighbors, that is, neighbors in the plane but not in attribute space: increasing dimension by one allows to get rid of some of the false neighbors.
3. Results 3.1. Random graphs with clusters The algorithm was tested on random graphs Gðn; c; pi ; pe Þ [16], where n is the number of vertices per cluster, c is the number of clusters, pi is the probability that two vertices within a cluster are connected, and pe is the probability that two vertices belonging to different clusters are connected. Let us set for simplicity eij ¼ 1 if ðvi ; vj Þ 2 E and eij ¼ 0 if ðvi ; vj Þ 62 E. Training vectors xm are presented according to a uniform distribution over the portion of plane considered ð½0; 1 ½0; 1 Þ. Initially, all vertices are randomly placed in this area. In order to avoid some vertices oscillating back and forth on the plane because they are subject to conflicting constraints, bðmÞ is modified when m > T =3 : bðmÞ ¼ expð kwj wiðmÞ k=dc Þ for vj 2 N 1 ðviðmÞ Þ, where dc is a characteristic distance beyond which it is pointless to drag nodes. Fig. 1 illustrates the fact that the algorithm is able to find ‘‘natural’’ clusters in a graph Gð25; 4; 0:8; 0:01Þ: Fig. 1(a) shows the initial random distribution of nodes in ½0; 1 ½0; 1 ; Fig. 1(b) shows that, after T ¼ 2000 training vectors, vertices distribute themselves in space in such a way that exactly c ¼ 4 clusters of vertices appear, that correspond to the four clusters of the graph. Note that the application of a clustering algorithm to the vertex positions obtained after
164
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 2. (a) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 1(a). k k is the L1 norm in the plane, and dð Þ is the L1 norm for graphs. (b) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 1(b).
E. Bonabeau / Information Sciences 143 (2002) 159–180
165
T ¼ 2000 training vectors would yield an optimal partitioning of the graph: the SOM-based algorithm is indeed a very efficient approach to partitioning [4]. The algorithm can now be evaluated as a mapping from graph space onto the plane, that is, as a MDS algorithm: we expect nodes which are far apart (respectively close) in the graph to lie far away from (respectively close to) each other on the plane, and vice versa. Let us introduce the L1 distance dðvi ; vj Þ between two nodes in a graph: n X d vi ; vj ¼ n 1 eik ejk :
ð5Þ
k¼1
In the present case, where eij ¼ 0 or 1, we have dðvi ; vj Þ ¼ n 1 jDðqðvi Þ; qðvj ÞÞj, where qðvi Þ ¼ fvj 2 V ; eij 6¼ 0g is the set of edges adjacent to vi , including vi , D is the symmetric difference ðDðA; BÞ ¼ ðA [ BÞ ðA \ BÞ), and j j denotes the number of elements of a set. The use of D expresses the fact that vertices that possess many common adjacent edges are close in the graph, whereas those with only a few or no adjacent edges in common are far apart in the graph. In a random graph of the type studied here, we expect to have two characteristic distances in the graph: the distance between vertices that belong to the same cluster, and the distance between vertices that do not belong to the same cluster. Fig. 1(a) shows kwi wj k as a function of dðvi ; vj Þ for the same graph and the same distributions of vertex positions in the plane as in Fig. 1(a): the two expected clouds of points are observed, and there is, of course, no significant correlation between kwi wj k and dðvi ; vj Þ. Fig. 2(b) shows the same relationship after T ¼ 2000 training vectors (vertex positions are those of Fig. 1(b): the distance between the positions of vertices in the plane representation is strongly positively correlated to their distance in the graph (P < 0:005), with two clouds of points corresponding to the two characteristic distances in the graph. As the reader can easily convince himself, it is not possible to build a general 2d isometric embedding of a graph with more than three nodes; nevertheless the algorithm is able to generate a good mapping of the graph onto the plane. The choice of the L1 norm in the design of the SOM-based algorithm rather than another norm, such as the Euclidian norm, was dictated by the finding that MDS using a Euclidian norm does not lead to a reliable detection of a graph’s clusters [13]. Hubert and Arabie [19] studied MDS algorithms with the L1 norm, concluded that gradient-based techniques were inadequate and proposed a more sophisticated technique which, because of its complexity, was applied to relatively small data sets. The SOM-based method, although it is an implicit MDS method, can be applied to large data sets. Fig. 3 shows the representation of a graph Gð25; 4; 0:8; 0:01Þ in three dimensions: here, wi 2 ½0; 1 ½0; 1 ½0; 1 and training vectors xm are presented according to a uniform distribution over ½0; 1 ½0; 1 ½0; 1 . Again, the four clusters can be clearly visualized, although not better than in two dimensions
166
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 3. Representation of a graph Gð25; 4; 0:8; 0:01Þ in three dimensions. wi 2 ½0; 1 ½0; 1 ½0; 1 and training vectors xm are presented according to a uniform distribution over ½0; 1 ½0; 1 ½0; 1 . Vertices distribute themselves in space in such a way that exactly c ¼ 4 clusters of vertices appear, that correspond to the four clusters of the graph, but visualization is not better than in two dimensions (Fig. 1(b)). Same parameter values as in Fig. 1(b).
(Fig. 1(b)). Fig. 4 shows kwi wj k as a function of dðvi ; vj Þ for the graph representation of Fig. 3, where k k is the L1 norm in R3 : Fig. 4 is similar to Fig. 2(b). 3.2. Random geometric graphs Fig. 5(c) shows the representation of a randomly generated geometric graph with 200 vertices, shown on Fig. 5(a), found by the algorithm. In geometric graphs, which are used in the context of numerical modeling and finite element simulation, vertices, characterized by Euclidean coordinates, are located on a plane and are locally connected to other vertices that lie within a radius of connectivity R (here, R ¼ 0:18). Vertices are initially randomly distributed in the unit square (Fig. 5(b)). We set f ðmÞ ¼ 3 for 1 6 m 6 T ¼ 2000 to increase the range of influence of nodes. The unfolding of the graph to approximate the training vector distribution leads to a remarkable representation where vertices are located on the plane in such a way that connections are local. Fig. 6(a) shows kwi wj k as a function of dðvi ; vj Þ for the same graph and the same distribution of vertex positions in the plane as in Fig. 5(a): one cloud of points is observed and there is no correlation between kwi wj k and
E. Bonabeau / Information Sciences 143 (2002) 159–180
167
Fig. 4. kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 3. k k is the L1 norm in R3 and dð Þ is the L1 norm for graphs.
dðvi ; vj Þ. Fig. 6(b) shows the same relationship after T ¼ 2000 training vectors (vertex positions are those of Fig. 5(b)): the distance between the positions of vertices in the plane representation is strongly positively correlated to their distance in the graph. Here again, the algorithm has done a good job at performing a form of MDS: for a specialist of geometric graphs, this is obvious upon visual inspection of Fig. 5(c). 3.3. Topological organization of the primate cortical visual system Young [31] studied the parcellation and connections of the primate cortical visual system and gave an ‘‘objective’’ representation of its topological organization using MDS. In his 2d visualization, points representing areas that have very similar afferent and efferent cortico-cortical connections are close together, whereas points representing areas that have very different patterns of connections are far apart. The entries of the connectivity matrix
168
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 5. (a) Geometric graph of 200 vertices in a square area of linear size 1, where the radius of connectivity is R ¼ 0:18 (L1 norm in the plane). (b) Initial distribution of vertices. (c) Representation of the graph found by the algorithm at T ¼ 2000. Same parameters as Fig. 1(b) except that f ðmÞ ¼ 3.
he used take on three values: eij ¼ 2 for reciprocal connections between areas i and j, eij ¼ 1 for one-way connections, and eij ¼ 0 for connections that have been tested for and found absent or connections which are not known. As a test of success of MDS, he used the entries of the connectivity matrix as a measure of the ‘‘closeness’’ of two areas: the aim of MDS was then to obtain a representation such that distances between points of the representation are as close as possible to the reverse rank order of the ‘‘proximities’’ between areas in the matrix. The problem with this is the small number of distance values (three values) in the connectivity matrix. Using the L1 norm for graphs, which is consistent with the problem at hand,
E. Bonabeau / Information Sciences 143 (2002) 159–180
169
Fig. 6. (a) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 5(b). (b) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 5(c).
170
E. Bonabeau / Information Sciences 143 (2002) 159–180
one can obtain a better validation of the approach, although the resulting structure is remarkably similar to the one obtained by Young [31]. Fig. 7 shows the 2d representation obtained after application of the SOM-based approach. As reported by Young [31], several organizational features of the connectional topology are apparent in Fig. 7: the two dimensions of the structure correspond roughly to the anterior–posterior (left to right in Fig. 7) and to the dorsal–ventral (bottom to top) spatial distribution of the areas in the brain. A more detailed description of the structure can be found in [31]. Fig. 8(a) shows kwi wj k as a function of eij , and a significant negative correlation is observed (P < 0:001), as expected. This relationship was the measure of success used by Young [31]. Despite the significance of the negative correlation, it is clear that there are too few values of eij for the interpretation to be unquestionable. Fig. 8(b) shows kwi wj k as a function of dðvi ; vj Þ for the same graph representation: here again, there is a significant correlation (though now positive, of course) (P < 0:001), but this time the interpretation is made easier by the fact that d can take on many more values. Fig. 9 shows a representation of the same graph when the distribution of training vectors is no longer uniform over ½0; 1 ½0; 1 but uniform over
Fig. 7. Representation of a graph given by a connectivity matrix which can be found in [31]. Same parameters as in Fig. 1(b). Node labels (cortical areas): 1: V1, 2: V2, 3: V3, 4: Vp, 5: V3a, 6: V4, 7: VOT, 8: V4t, 9: MT, 10: MSTd, 11: MSTI, 12: FST, 13: PITd, 14: PITv, 15: CITd, 16: CITv, 17: AITd, 18: AITv, 19: STPp, 20: STPa, 21: TF, 22: TH, 23: PO, 24: PIP, 25: LIP, 26: VIP, 27: DP, 28: A7a, 29: FEF, 30: A46.
E. Bonabeau / Information Sciences 143 (2002) 159–180
171
Fig. 8. (a) kwi wj k vs eij for the graph representation of Fig. 7. (b) kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 7.
172
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 9. Representation of the same graph as the one represented in Fig. 7. Same parameters as in Fig. 1(b) except that training vectors xm are uniformly distributed over ð½0; 0:4 ½0; 1 Þ[ ð½0:6; 0:1 ½0; 1 Þ.
Fig. 10. Representation in three dimensions of the same graph as the one represented in Fig. 7. wi 2 ½0; 1 ½0; 1 ½0; 1 and training vectors xm are presented according to a uniform distribution over ½0; 1 ½0; 1 ½0; 1 . Same parameter values as in Fig. 1(b).
E. Bonabeau / Information Sciences 143 (2002) 159–180
173
½0; 0:4 ½0:6; 1 , which leads to an implicit bi-partitioned representation of the graph with a ‘‘no-node’s land’’ in the 0:4 < x < 0:6 band. Application of this biased distribution of training vectors leads to a segregation of areas according to the anterior–posterior axis. Finally, Fig. 10 shows a 3d representation of the same graph. Although node labels are not represented for the sake of clarity, it appears clearly that the structure is quasi-2d, with only about 20% nodes located out of the planar component. It is likely that this near-planar (somewhat circular) structure reflects a system that is divided into two gross streams, which possess a common origin in the occipital cortex and reconverge after somewhat segregated processing [32]. Fig. 11 shows kwi wj k as a function of dðvi ; vj Þ for the graph representation of Fig. 10, where k k is the L1 norm in R3 : Fig. 11 is similar to Fig. 8(b), certainly because the structure possesses a strong 2d component, so that a 3d representation does not add a significant amount of information.
Fig. 11. kwi wj k vs dðvi ; vj Þ for the graph representation of Fig. 10. k k is the L1 norm in R3 and dð Þ is the L1 norm for graphs.
174
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 12. Representations of a work network in a technical laboratory. Algorithm parameters are identical to those of Fig. 1(b). (a) Only ties that correspond to daily interactions are taken into account. (b) Ties that correspond to daily and weekly interactions are taken into account. (c) Ties that correspond to daily, weekly and monthly interactions are taken into account. (d) Ties that correspond to daily, weekly, monthly and half-yearly interactions are taken into account. (e) Ties that correspond to daily, weekly, monthly, half-yearly and yearly interactions are taken into account. Data used by permission of Valdis Krebs (www.orgnet.com).
In addition to these results, it is worth remembering that the SOM-based approach to MDS is a force-directed algorithm: vertices exert forces to drag their neighbors with them as they move. It is not absurd to think that the genesis of the cortical topological organization may rely on a similar mechanism: neurons move toward certain areas while attracting neurons they are connected to through physical forces, axons being real physical objects. It this were to be the case, the algorithm would not only allow us to analyze the
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 12. (continued).
175
176
E. Bonabeau / Information Sciences 143 (2002) 159–180
Fig. 12. (continued).
connectivity structure, it would also replicate the very formation of the structure! 3.4. Network of workers in a technical laboratory I have applied the MDS technique to the visualization of a network of work relationships in a technical laboratory. The network involves 55 people. The bidirectional ties were determined through a survey and sorted according to the frequency of the interactions between the workers (daily, weekly, monthly, half-yearly and yearly). Figs. 12(a)–(d) show representations of the network when certain ties only are taken into account: ties that correspond to daily interactions for Fig. 12(a), daily and weekly for Fig. 12(b), daily to monthly for Fig. 12(c), and daily to half-yearly for Fig. 12(d). Several ‘‘hubs’’ appear in Fig. 12(a): employees 36, 52 and 55 are central in this network, especially 55. When weekly interactions are taken into account, one, surprisingly, observes a ‘‘spontaneous’’ partitioning of the network into two sub-networks and nodes 27, 42, 48 and 55 are located between the two subnetworks (Fig. 12(b)). While node 55 still seems to be central, nodes 27 and 42 have replaced nodes 36 and 52. When monthly interactions are taken into account, the same partitioning phenomenon is apparent (Fig. 12(c)), with node 19 lying in the middle and a set of other nodes (3, 18, 23, 25, 32, 39, 43, 45 and perhaps 53) located at the
E. Bonabeau / Information Sciences 143 (2002) 159–180
177
periphery of the structure: the nodes at the periphery lie there because they are weakly connected to the rest of the network. When half-yearly interactions are taken into account, the partition is no longer apparent (Fig. 12(d)), but there still are nodes that lie at the periphery, most of them identified in Fig. 12(c), plus nodes 8, 16 and 28. Including yearly interactions does not significantly modify the picture obtained in Fig. 12(d). Interpreting these results is of course not easy, but applying the SOM-based algorithm clearly allowed us to identify key properties of the network. Since the networks represented in Figs. 12(a) and (b) are certainly the most ‘‘relevant’’ to the understanding of the day-today work dynamics, the representations tell us that some nodes are particularly central, and that there may exist two ‘‘natural’’ (and yet unplanned) groups of people, with many intra-group interactions and significantly fewer inter-group interactions. 3.5. Food webs Finally, the approach has been applied to particular ecological networks called community food webs, which include all the species or all the kinds of
Fig. 13. Representation of the California tidal flat community food web [6]. The algorithm parameters are identical to those of Fig. 1(b). The entry matrix is not symmetric. Node labels (species or groups of organisms): 1—primary producers, 2—detritus, 3— protozoa, 4—crustacea, 5— Callianassa, 6—shore crabs, 7—Phoronopsis, 8—Urechis, 9—clams, 10—Lumbrinereis, Notomastus, 11—heron, 12—Clevelandia, 13—wading brids, 14—stingray, 15—Nereis, Glycera, 16—nemerteans, 17—terns, shorebirds, 18—cabezon, 19—flounder, 20—Cerebratulus, 21—loon, cormorant, 22—striped bass, 23—scoter, 24—gulls, 25—man.
178
E. Bonabeau / Information Sciences 143 (2002) 159–180
organisms (there may be several classes of organisms within a species—stage of life, size, etc.—or several taxonomically related species may be represented by the same node in the web) found in a particular habitat, and trophic relationships between them, that is, ‘‘who eats whom’’ [6,8]. Food webs are obviously directed graphs, which suggests that connections should not be treated as symmetric. Here only mother-vertices (with outgoing edges) drag daughter-vertices (with incoming edges) when they move: vertices with outgoing edges are predator species and vertices with incoming edges are species predated upon. In other words predator species drag prey species. Predator species that are not predated upon are expected to lie at the periphery of the unit square. Fig. 13 shows a representation of the California tidal flat food web, the adjacency matrix of which can be found in [6]. The matrix, which has entries for 25 species including man, is binary: eij ¼ 1 if species j is a predator of species i, and eij ¼ 0 otherwise. Fig. 14 shows kwi wj k as a function of eij , and a significant negative correlation is observed (P < 0:001). The representation reveals, as expected, major predator species: the corners of the unit square are
Fig. 14. kwi wj k vs eij for the graph representation of Fig. 13.
E. Bonabeau / Information Sciences 143 (2002) 159–180
179
occupied by man (#25), birds (#11,#13 and #21). Keystone species can also be seen: they are species located at the center of the representation with many connections (examples are #2 and #6). In summary the modified algorithm has been able to highlight the structure of the food web. Treating connections as symmetric would not allow to reveal the predator–prey structure but would still clearly allow to observe the keystone species.
4. Conclusion The SOM-based approach to graph drawing amounts to performing a form of MDS on graphs which may also explain why it is so efficient at partitioning [4]. Future work will be aimed at making the connection between MDS and SOM-based graph drawing more rigorous and at testing different kinds of distance functions other than the L1 distance used in the present paper. This approach is very easy to implement and works particularly well on large graphs because a large graph can more easily ‘‘learn’’ the distribution of training vectors: it can therefore be used in many situations for exploratory network data analysis.
Acknowledgements My work is supported by the Interval Research, Inc. Fellowship in Adaptive Computation at the Santa Fe Institute. I would like to thank Valdis Krebs for insightful discussions and for providing me with the technical lab data.
References [1] G.D. Battista, P. Eades, R. Tamassia, I.G. Tollis, Algorithms for drawing graphs: an annotated bibliography, Comput. Geom. 4 (1994) 235–282. [2] B. Becker, G. Hotz, On the optimal layout of planar graphs with fixed boundary, SIAM J. Comput. 16 (1987) 946–972. [3] E. Bonabeau, F. Henaux, Self-organizing maps for drawing large graphs, Inf. Process. Lett. 67 (1998) 177–184. [4] E. Bonabeau, F. Henaux, Monte Carlo partitioning of graphs using self-organizing maps, Int. J. Mod. Phys. C 9 (1998) 1107–1119. [5] F.J. Brandenburg, M. Himsolt, C. Rohrer, An experimental comparison of force-directed and randomized graph drawing algorithms, in: Proceedings of the Graph Drawing ’95, Lecture Notes in Computer Science, 1027, 1995, pp. 76–87. [6] F. Briand, Environmental control of food web structure, Ecology 64 (1983) 253–263. [7] R.-I. Chang, P.-Y. Hsiao, VLSI circuit placement with rectilinear modules using three-layer force-directed self-organizing maps, IEEE Trans. Neural Networks 8 (1997) 1049–1064. [8] J.E. Cohen, F. Briand, C.M. Newman, Community Food Webs, Springer, Berlin, 1990.
180
E. Bonabeau / Information Sciences 143 (2002) 159–180
[9] M.K. Coleman, D.S. Parker, Aesthetics-based graph layout for human consumption, Software – Pract. Exper. 26 (1996) 1–25. [10] T.F. Cox, M.A.A. Cox, Multidimensional Scaling, Chapman & Hall, London, 1994. [11] R. Davidson, D. Harel, Drawing graphs nicely using simulated annealing, ACM Trans. Graphics 15 (1996) 301–331. [12] P. Eades, A heuristic for graph drawing, Congr. Numer. 42 (1984) 149–160. [13] H. de Frayssex, P. Kuntz, Pagination of large-scale networks; embedding a graph in Rn for effective partitioning, Algorithms Rev. 2 (1992) 105–112. [14] A. Frick, A. Ludwig, H. Mehldau, A fast adaptive layout algorithm for undirected graphs, in: Proceedings of the Graph Drawing ’94, Lecture Notes in Computer Science, 894, 1994, pp. 388–403. [15] T.M.J. Fruchterman, E.M. Reingold, Graph drawing by force-directed placement, Software – Pract. Exper. 21 (1991) 1129–1164. [16] J. Garbers, H.J. Pr€ omel, A. Steger, Finding clusters in VLSI circuits, in: A. SangiovanniVincentinelli, S. Goto (Eds.), Proceedings of the IEEE International Conference on ComputerAided Design, IEEE Computer Society Press, Silver spring, MD, 1990, pp. 520–523. [17] A. Hemani, A. Postula, Cell placement by self-organization, Neural Networks 3 (1990) 377–383. [18] J. Hertz, A. Krogh, R.G. Palmer, Introduction to the Theory of Neural Computation, Addison-Wesley, Redwood City, CA, 1991. [19] L. Hubert, P. Arabie, Multidimensional scaling in the city-block metric; a combinatorial approach, J. Classification 9 (1992) 211–236. [20] A.K. Jain, R.C. Dubes, Algorithms for Clustering Data, Prentice-Hall, Englewood Cliffs, NJ, 1988. [21] T. Kamada, Visualizing Abstract Objects and Relations, World Scientific, Singapore, 1989. [22] T. Kamada, S. Kawai, An algorithm for drawing general undirected graphs, Inf. Process. Lett. 31 (1989) 7–15. [23] B. Kernighan, S. Lin, An efficient heuristic procedure for partitioning graphs, Bell Syst. Tech. J. 49 (1970) 291–307. [24] T. Kohonen, Self-organization and associative memory, Springer, Berlin, 1989. [25] T. Kohonen, The self-organizing map, Proc. IEEE 78 (1990) 1464–1480. [26] P. Kuntz, D. Snyers, P. Layzell, A stochastic heuristic for visualising graph clusters in a bidimensional space prior to partitioning, J. of Heuristics 5 (1999) 327–351. [27] T. Lengauer, Combinatorial algorithms for integrated circuit layout, Wiley, Stuttgart, 1990. [28] N. Quinn, M. Breur, A force directed component placement procedure for printed circuit boards, IEEE Trans. Circuits and Systems CAS-26 (1979) 377–388. [29] R. Tamassia, G. Battista, C. Battini, Automatic graph drawing and readability of diagrams, IEEE Trans. Syst. Man Cybern. 18 (1988) 61–79. [30] W.T. Tutte, How to draw a graph, Proc. London Math. Soc. 13 (1963) 743–768. [31] M.P. Young, Objective analysis of the topological organization of the primate cortical visual system, Nature 358 (1992) 152–155. [32] M.P. Young, Scaling and brain connectivity, Nature 369 (1994) 449–450. [33] C.-X. Zhang, D.A. Mlynski, Mapping and hierarchical self-organizing neural networks for VLSI placement, IEEE Trans. Neural Networks 8 (1997) 299–314.