- Email: [email protected]
The knowledge base partitioning problem: Mathematical formulation and heuristic clustering
Data & Knowledge Engineering 8 (1992) 329-337 North-Holland
329
The knowledge base partitioning problem: Mathematical formulation and heuristic clustering Tzvi R a z and N a n c y A. B o t t e n IBM Corporation, Westlake Software Development Laboratory, Roanoke, TX 76299, USA Abstract Raz, T. and N.A. Botten, The knowledge base partitioning problem: Mathematical formulation and heuristic clustering, Data & Knowledge Engineering 8 (1992) 329-337. Allocation of production rules among several partitions in a knowledge base can shorten the compilation and execution of expert systems applications and facilitate their verification, validation ad maintenance. The problem of allocation of rules or groups of rules among partitions of limited size while minimizing the sum of inter-partition connections is formulated as a 0-1 integer programming problem with a quadratic objective function. Since this type of problems is NP-complete, a clustering algorithm based on the nearest neighbor heuristic is proposed.
Keywords. Knowledge base design; rule base programming; graph partitioning.
1. Introduction
The design of the knowledge base for an expert system involves acquiring the knowledge from one or possibly several domain experts, and coding it in a form compatible with the software environment of the system. Quite often, not all the knowledge is applicable to and required by each and every case for which the expert system might be used. It is also possible that substantial advantages in memory requirements and execution time may be achieved if the knowledge base is divided into a number of chunks or partitions, such that only one of them is loaded at any time. Since the partitions contain only a fraction of the knowledge in the system, each partition is less complex that the entire system. The reduction in complexity obtained by partitioning the knowledge base may facilitate its verification and validation in the design and coding stages, and its maintenance and updating throughout the life cycle of the system. Other potential benefits of knowledge partitioning include the possibilities of development in parallel and of reusability of coded knowledge. This discussion provides the motivation for allocating the knowledge contained in an expert system among a number of partitions. Of course, partitioning, like everything else, does not come free. Its costs derive from two main sources: additional overhead and switching among partitions. The cost of additional overhead is borne by the user, who has to remember what parts of the knowledge are kept in each partition and may have to switch partitions in the course of a session with the expert system. The overhead problem may be alleviated to some extent by good software design, especially in the user interface, but an expert system consisting of several knowledge bases will always be somewhat more complicated than one consisting of a single knowledge base. However, many users will gladly pay 0169-023X/92/$05.00 © 1992- Elsevier Science Publishers B.V. All rights reserved
330
7". R a z . N . A .
Botten
the price of additional overhead it it brings about a significant reduction in compilation and execution time. The cost of switching among partitions is directly related to the extent of reasoning connections among the rules in the different partitions of the knowledge base. A connection between two partitions exists if a line of reasoning requires knowledge stored in both of them in order to reach a conclusion useful to the user. Connections may be best explained in terms of production rules, which are among the most common forms of knowledge representation in expert systems. A production rule consists of premise part and a conclusion part. The premise contains a pattern of elements, which, if matched by the facts stored in the working memory of the expert system, causes the elements in the conclusion part to be executed. A connection between two partitions exists if one of the partitions contains a rule with a conclusion element that is a premise element in a rule that belongs to the other partition, or vice versa. A connection between partitions has a negative effect on the performance of the expert system. This is due to the fact that the partition connected to must be loaded (and, depending on the environment, may also need to be compiled), replacing the currently loaded partition while preserving the current contents of the working memory. This operation requires a relatively large amount of time, slowing down the response time of the expert system. Obviously, it is in our interest to partition the knowledge base in a way that minimizes the extent of switching among partitions. This paper deals with the problem of knowledge base partitioning in the context of rule-based expert systems. Following a review of the relevant literature, we present a mathematical formulation of the problem. Since the problem turns out to be NP-complete, we propose a heuristic clustering algorithm for knowledge base partitioning, and illustrate its application in an actual case study.
2. Review of the literature
Grouping of rules according to subject or use in order to assist designers, developers and users is advocated by many reference and text books on expert systems. For instance, Hayes-Roth et al. [10] and Martin and Oxman [16] mention the use of metarules (rules containing knowledge about the organization of the knowledge base) as a means for navigating among the various rule groups, and Cooper and Wogrin [6] devote a chapter of their text on rule-based programming to the implementation and control of rule clusters for improving the performance of OPS5 programs. The following papers addressing explicitly the issue of knowledge base partitioning are discussed in some detail. Jacob and Froscher [11] addressed the problem of production rule-base partitioning from the perspective of facilitating maintenance of the knowledge base. Their objective was to divide the rules into groups to reduce the amount of information that a knowledge engineer must understand in order to make changes, while limiting the flow of information among the groups. Rules are related to each other through elements the values of which they use or modify. The relationships can be described by a graph, with nodes representing rules and edges representing common elements. These authors presented two approaches to knowledge base partitioning. This first consists of dividing the rules into groups so that each group produces only one fact that is used by other groups. The second approach is a clustering algolJthm based on a measure of closeness between rules. The closeness between two rules is the weighted sum of their common elements. The weights reflect whether the element appears in both rules as a condition (0.33); as a conclusion (0.5) or as a condition in one rule and as a conclusion in the other (1.0). The closeness between a rule and a group of rules is the average of the closeness measures between the rule and each
The knowledge base partitioning problem
331
of the individual rules that belong to the group. Clustering proceeds by iteratively combining the closest pair of rules or rule groups and updating the closeness values. The authors reported that encouraging results were obtained in the analysis of several existing knowledge bases. Cheng and Fu [4] presented a clustering procedure for discovering the underlying concepts in a knowledge base consisting of production rules. The distance between two production rules is defined as the normalized number of elements not common to both. Rules groups ('stars') are created by bringing together the rules whose distance from the rule 'seed' is below a statistically determined threshold. The rule groups are replaced by representative rules, which contain the most frequent rule elements found in the group. The process is repeated until no additional changes result. At that point the elements of the representative rules constitute the underlying concepts of the knowledge base. The published literature containing quantitative techniques for knowledge base partitioning is relatively scarce. Several other references containing generic approaches or dealing with software partitioning in general are also available. The interested reader is directed to the comprehensive review prepared by Botten, Kusiak and Raz [1].
3. Problem statement
The knowledge base partitioning problem consists of finding the optimal assignment of production rules to a number of partitions. Typically the development of the knowledge base proceeds in a top-down manner by decomposing the domain into successively smaller topical areas, up to the point where the knowledge relevant to each of these areas can be expressed by a relatively small number of production rules. Since the rules within each such area are closely related, by grouping them together and assigning them to partitions as a group, we may reduce the need to switch among partitions in the course of a consultation session with the expert system. By committing to maintain the topical areas intact, the problem is redefined as the assignment of topical areas to partitions. Of course, there is no loss of generality in this approach, since if we wish to consider each rule in separate, then we define each topical area as containing exactly one rule. Two types of constraints are present. First, the assignment must be exhaustive and mutually exclusive, meaning that each and every topical area must be assigned to exactly one partition. Second, the size (in terms of the number of rules) of the partitions should not exceed a given value. The objective function to be optimized reflects the extent to which the partitions are independent of each other, as measured by the number of connections across partitions. A connection is defined as an instance of a rule with a premise element in one partition and a conclusion element in a different partition. We may assign different weights to the connections based on the number of elements involved, on the likelihood that the rule will be invoked in a session with the expert system, or on a subjective measure of the desirability of keeping the rules in the same partition. In any case, the objective of the problem is to find the assignment that minimizes the weighted sum of the connections across partitions.
4. Mathematical programming formulation
In this section we develop a mathematical programming formulation of the knowledge base partitioning problem. There are N areas to be assigned among M partitions. The following notation is used.
T. Raz, N . A . Botten
332
x~k = 1 if area i is assigned to partition k 0 otherwise a~j = weight of the connection between area i and area j A = adjacency matrix containing the weights of the connections between areas (a~j) bi = size of area i, in terms of number of rules or some other relevant measure B = maximum partition size. The smallest number of partitions that permits a feasible solution is the smallest integer not smaller than (Zi~ ~ bi)/B. However, this number does not assure the existence of a feasible solution. If the size of several areas is close to the partition size, then it may not be possible to fit more than one partition per area. For example, consider four areas with three rules each. Partition size cannot exceed four rules. Although in principle three partitions would contain the 12 rules in the system, in this case it is obvious that three would not suffice, and four are needed to obtain a feasible solution. To avoid this problem, one may formulate the problem with more partitions than the absolute minimal number, keeping in mind that the optimization procedure will ensure that unneeded partitions will remain empty. For any partition k the quantity Xik(1 -- Xjk ) is equal to 0 if and only if the two areas i a n d j belong to partition k, and is equal to 1 if area i belongs to partition k and area j belongs to a different partition. Consequently, the objective function may be formulated as: M
N
N
~ ~ Xik(l-
MIN ~ k=l
Xjk)aij
(1)
i=l j-I
Note that the triple summation yields twice the sum of the weights of the connections across partitions, since each connection is counted twice. Of course, this does not affect the solution. If, however, we prefer to deal directly with sum of the weights of the connections and reduce somewhat the number of calculations required, then we use the following objective function: M
N
i-1
~'~ ~ xig(1- xjk)aii
MIN ~ k=l
(2)
i=2 j=l
We may observe that the sum of all the weights of the connections among areas is constant. Consequently, minimization of the sum of the connections among partitions is equivalent to maximization of the sum of the connections within partitions. Thus we have an alternative objective function: M
MAX ~ k=l
N
~
i-1
~ XikXjka q
(3)
i-2 j=l
There are three sets of constraints: 1. The decision variables Xik must be 0 or 1: x~k E {0, 1)
for all areas i = 1 , . . . , N and all partitions k = 1 . . . . .
M
(4)
2. Each area must belong to exactly one partition: M
x~=l k=l
for all a r e a s i = l , . . . , N
(5)
The knowledge base partitioning problem
333
3. The partition sizes should not exceed the limit B: N
Xikb i <<-B
for all partitions k = 1. . . . . M
(6)
i=1
The three sets of constraints in equations (4), (5) and (6) are applicable regardless of which formulation (1), (2) or (3) we choose for the objective function. In any case we have an integer quadratic programming problem with M x N 0-1 decision variables, N equality constraints and M inequality constraints. In a series of papers, Picard and Ratliff [18-20] established the equivalence between 0-1 integer programming with a quadratic objective function and the graph partitioning problem. This problem has received a fair amount of attention in the literature, which we review briefly next. Gorinshteyn [9] developed a branch and bound method to the graph partitioning problem when all the nodes are of size 1. Kernighan and Lin [13] developed a heuristic for minimizing the sum of the weights on the cut-sets when all the nodes are of equal size. Later on, Kernighan [12] developed an exact optimization algorithm for a variation of that problem with the additional constraint that the assignment of nodes to subsets must preserve a linear order of the nodes. Christofides and Brooker [5] proposed a tree search method for the optimal partitioning of a weighted graph into two parts, using the concept of the longest spanning tree to direct the search, with bounds calculated with a maximum flow algorithm. The version of Minoux and Pinson [17] of the graph partitioning problem has a constraint on the number of subsets rather than on the subset size. The algorithm of Duncan and Scott [7] for partitioning a weighted graph with equal sized nodes in order to minimize the interset weights is based on a probabilistic branch and bound approach. Rutkowski [21] approached the graph partitioning problem by tearing nodes instead of disconnecting edges. His solution is based on heuristics for permutating the rows and columns of the fundamental circuit matrix corresponding to a tree in the graph. Lawler, Levitt and Turner [15] used graph partitioning to solve the problem of allocation of electronic gates to modules (subsets) of limited size in order to minimize transmission delay for the entire network. Their approach consisted of a heuristic clustering algorithm designed to minimize the length of the longest path in a directed graph, with the distance between gates being defined as the number of modules that the signal had to travel through. Breuer [3] also studied the electronic circuit partitioning problem and developed three heuristic placement algorithms based on finding the minimal cut in the circuit graph for minimizing transmission path length. Krishnamurthy [14] expanded the work of Kernighan and Lin [13] as it applies to VLSI network design and developed improved heuristics for selecting the pair of nodes to interchange. None of the variations of the graph partitioning problem solved is directly applicable to the knowledge partitioning problem, which has the following characteristics: the graph is not directed and its nodes are not ordered; both nodes and edges have weights; the subset size is constrained from above; and the objective function is the sum of the weights of the cut-set edges. In fact, this particular type of problem appears in the list of NP-complete problems compiled by Garey and Johnson [8]. Although small instances of the problem may be solved with the assistance of specialized software, in general the computational time and resources required are quite large and increase exponentially with the number of topical areas and partitions. This situation justifies the use of heuristics for solving the knowledge base partitioning problem.
T. Raz, N.A. Botten
334
5. Heuristic clustering In the following section we present a clustering approach based on the nearest neighbor, or greedy, heuristic: we attempt to assign to the same partition the areas that are most closely connected. The areas are combined into clusters in descending order of their connection weights air This continues until the number of clusters equals the desired number of partitions, while skipping over those combinations that result in cluster size larger than the given limit on the partition size. This algorithm operates on the adjacency matrix A, which contains the weights of the connections between areas (a~j). At each iteration, some of the values in the matrix A are changed to keep track of the areas that were considered for combination. The algorithm is summarized next.
Initialization: 1. Each area is a cluster 2. a , ~ 0 for all i (ignore the weight of the connections within areas) Do While Not all aij = 0 Step 1. Find the two clusters i' and j' (i' < j ' ) with the largest connection weight between them (ties are decided arbitrarily) ai, r = MAX aq ij~[1, N]
a t / , , a j, i, ~ O (remove connection from further consideration)
Step 2. Check for feasibility of combination: If b i, + b r <~ B (combination feasible) then proceed to Step 3 Otherwise, go to Step 1
Step 3. Combine cluster i' and cluster j': For all k: ark ~ ai,~ + aj, k
a/,k ~ 0 aki, ~ aki, + akj,
akj, ~ 0
(combine row of ]' with row of i') (remove from further consideration the combined row) (combine column of j' with column of i') (remove from further consideration the combined column)
Go to Step 1
E n d Do While The algorithm can have at most N iterations. The dominant calculation in each iteration consists of finding the maximum of at most N 2 aij values, which requires N2-1 comparisons. By taking into account the fact that the A matrix is symmetric with respect to its diagonal, the number of comparisons may be reduced by half. Additional reductions in the number of comparisons may be obtained by reducing the order of A as the iterations progress. However, these will not bring a reduction in the order of magnitude. Overall, the computational complexity of the algorithm is of the order of N * N 2, or O(Na). The computational effort required to obtain the adjacency matrix A should also be considered.
The knowledge base partitioning problem
335
Defining the weights of the connections a, as a function of the number of connections between area i and area j, the values of a 0 are obtained by comparing the premise part of every rule to the conclusion part of every rule not in the same area. The resulting computational complexity is of order O(K2), where K is the total number of rules in the knowledge base. We note that the complexities of both the clustering algorithm and the setting up of matrix A are low-order polynomials.
6. Example The data for this example is taken from a project that involved the design of a knowledge base with 178 rules divided into 15 topical areas. The maximum allowed partition size was set equal to 50 rules. Full details of the case appear in Botten and Raz (1990). The entries a 0. of the adjacency matrix A represent the number of rules in area i that either have a premise (IF) element the value of which is affected by an action (THEN) element of a rule in area j, or have an action element that affects the value of a premise element of a rule in area j. Based on this definition, aij = a~ and A is symmetric. Since internal connections within areas are ignored, a , = 0 for all i. The adjacency matrix and the number of rules in each area appear in Fig. 1. At the beginning each area is a separate cluster. We identify the two clusters with the largest connection weight, which are cluster 3 and 4, with a34 -- a43 = 5. In the first iteration, these two clusters are combined, with the result that cluster 3 now contains areas 3 and 4, and all the entries involving area 4 have been set equal to 0. Figure 2 shows the resulting adjacency matrix and cluster sizes, with the values affected by the iteration in bold face. Now the two clusters with the largest connection weight between them (8) are cluster 3, which includes areas 3 and 4, and cluster 10, which so far includes area 10 only, and the two are combined. The results of the algorithm at each iteration are summarized in the dendogram in Fig. 3, which also shows the value of aii that was used for the combination and the size of the newly created cluster. Since the smallest number of partitions is 4, there were 1 5 - 4 = 11 iterations. The four final clusters, each corresponding to a partition of the knowledge base, are: {3, 4, 7, 9, 10}; {1, 2, 6, 12, 13}; {5, 8, 14}; and {11, 15}. The sum of the weights of the connections among the partitions was 31, which turned out to be equal to the optimal value found by enumeration. area
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
,1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 size
0 1 1 1 0 1 2 1 4 1 0 3 3 0 0 11
1 0 1 0 0 0 2 0 4 3 0 0 0 0 0 10
1 1 0 1 0 1 1 1 1 4 2 0 0 0 0 5
1 3 5 3 0 0 3 0 3 4 0 0 0 0 0 8
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9
1 0 1 0 0 0 0 0 0 0 0 0 1 0 0 4
2 2 1 2 0 0 0 2 4 3 0 0 0 0 0 11
1 0 1 0 0 0 2 0 0 0 0 0 1 1 0 16
4 4 1 4 0 0 4 0 0 3 0 0 1 0 0 5
1 3 4 3 0 0 3 0 3 0 0 0 0 0 0 18
0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 17
3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8
3 0 0 0 0 1 0 1 1 0 0 0 0 0 0 15
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 22
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19
Fig. 1. A d j a c e n c y matrix and cluster sizes before iteration 1.
336
T. Raz, N . A . Batten
area
5
6
7
8
9
10
11
12
13
14
15
1 3 8
0 0 2
3 0 0
3 0 0
0 0 0
0 0 0
0
1
0
1
2
0
2 3
1 2
0 4
4 0
0 0
0 0 0
1 0 I
2 2 4
I 0 I
4 4 4
4
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5 6 7 8 9 10 11 12 13 14 15 size
0 1 2 1 4 1 0 3 3 0 0 11
0 0 2 0 4 3 0 0 0 0 0 10
0 1 4 1 4 8 2 0 0 0 0 13
0 0 0 0 0 0 0 0 0 0 0 0
0
0
0
0
0
0
0
0
0
0
0 0 0 0 0 0 0 0 0 0 9
0 0 0 0 0 0 0 1 0 0 4
0 0 2 4 3 0 0 0 0 0 11
0 0 2 0 0 0 0 0 1 1 0 16
0 4 0 0 3 0 0 1 0 0 5
0 3 0 3 0 0 0 0 0 0 18
0 0 0 0 0 0 0 0 0 0 17
0 0 0 0 0 0 0 0 0 0 8
1 0 1 1 0 0 0 0 0 0 15
0 0 1 0 0 0 0 0 0 0 22
0 0 0 0 0 0 0 0 0 0 19
Fig. 2. Adjacency matrix and cluster sizes after iteration 1. Areo 3 4 I0 7 9 I 12 15 6 2 8' 14 5 11 15 Iteration Largest a~j
I]
I..............................
]- . . . . . . . . . . . . . . . . . . . . . .
t.....................
I I1 I2I3I4I5I 6 5 8 7 1 1 3 3 2
t 9t 10Jt11 12t 13t 14It 15
8
1
1
0
0
New Cluster Size 13 31 42 47 19 34 38 48 38 47 36
Fig. 3. Dendogram of the iterations of the clustering algorithm.
7.
Concluding
remarks
T h e d e v e l o p m e n t , use a n d m a i n t e n a n c e of a k n o w l e d g e base m a y be facilitated by its d i v i s i o n i n t o several p a r t i t i o n s . I n such a case, it is i m p o r t a n t to assign the rules or g r o u p s of rules to p a r t i t i o n s in a m a n n e r that m i n i m i z e s the n u m b e r of i n t e r p a r t i t i o n c o n n e c t i o n s . This t y p e of p r o b l e m is e q u i v a l e n t to the g r a p h p a r t i t i o n i n g p r o b l e m w h e n n o d e s a n d arcs have weights, t h e r e is a n u p p e r limit o n the p a r t i t i o n size a n d the o b j e c t i v e is to m i n i m i z e the s u m of t h e weights of the cut sets. I n this p a p e r we p r o p o s e d a clustering a l g o r i t h m b a s e d o n the n e a r e s t n e i g h b o r heuristic. I n o u r e x a m p l e , which is t a k e n f r o m a n actual e x p e r t system d e v e l o p m e n t p r o j e c t , the heuristic a l g o r i t h m yielded a n o p t i m a l solution. O b v i o u s l y , m o r e e x p e r i m e n t a t i o n is r e q u i r e d if o n e is i n t e r e s t e d in the p e r f o r m a n c e of the heuristic u n d e r d i f f e r e n t c o n d i t i o n s . H o w e v e r , we b e l i e v e that its low c o m p u t a t i o n a l c o m p l e x i t y , O ( N 3 ) ,
The knowledge base partitioning problem
337
and its intuitive logic make it an attractive alternative to solving the exact mathematical
formulation of the knowledge base partitioning problem (0-1 integer-programming with a quadratic objective function), which happens to be NP-complete. References [1] N. Botten, A. Kusiak and T. Raz, Knowledge bases: Integration, verification and partitioning, Europ. J. Operat. Res. 42 (1989) 117-128. [2] N. Botten and T. Raz, Knowledge base development for an expert system for civil rights investigation, Expert Systems 7 (1990) 224-235. [3] M.A. Breuer, Min-cut placement, J. Design Automation and Fault-tolerant Comput. 1 (1977) 343 -362. [4] Y. Cheng and K. Fu, Conceptual clustering in knowledge organization, IEEE Trans. Pattern Analysis Machine Intelligence. PAMI-7 (1985) 592-598. [5] N. Christofides and P, Brooker, The optimal partitioning of graphs, SIAM J. Applied Mathemat. 30(1) (1976) 55-69. [6] T.A. Cooper, and N. Wogrin, Rule-Based Programming with OPS5 (Morgan Kaufmann, San Mateo, CA, 1988). [7] J. Duncan and L.W. Scott, A branch-and-bound algorithm for pagination, Operat. Res. 23(2) (1975) 240-259. [8] M.R. Garey and D.S. Johnson, Computers and Intractability- A Guide to the Theory of NPCompleteness (W.H. Freeman, New York, 1979) [9] L.L. Gorinshteyn, The partitioning of graphs, Engrg. Cybernet. 1 (1969) 76-82. [10] F. Hayes-Roth, D.A. Waterman and D.B. Lenat, eds., Building Expert Systems (Addison-Wesley, Reading, MA, 1983). [11] R. Jacob and J. Froscher, Developing a software engineering methodology for rule-based systems 1985 Conf. on Intelligent Systems and Machines, Oakland University (1985) 179-185. [12] B.W. Kernighan, Optimal sequential partitions of graphs, J. Assoc. Comput. Machinery 18(1) (1971) 34-40. [13] B.W. Kernighan and S. Lin, An efficient heuristic procedure for partitioning graphs, Bell Syst. Tech. J. 49(2) (1970) 291-307. [14] B. Krishamurthy, An improved min-cut algorithm for partitioning VLSI networks, IEEE Trans. Comput. C-33(5) (1984) 438-446. [15] E.L. Lawler, K.N. Levitt and J. Turner, Module clustering to minimize delay in digital networks, IEEE Trans. Comput. C-18(1) (1969) 47-57. [16] J. Martin and S.W. Oxman, Building Expert Systems (Prentice-Hall, Englewood Cliffs, N J, 1988). [17] M. Minoux and E. Pinson, Lower bounds to the graph partitioning problem through generalized linear programming and network flows. RAIRO
[18]
[19] [20]
[21]
Recherche Operationelle/ Operat. R e s . 21(4) (1987) 349-364. J.C. Picard and H.D. Ratliff, A graph-theoretic equivalence for integer programming, Operat. Res. 21(1) (1973) 261-269. J.C. Picard and H.D. Ratliff, Minimum cuts and related problems, Networks 5 (1975) 357-370. J.C. Picard and H.D. Ratliff, A cut approach to a class of quadratic integer programming problems, Networks 10 (1980) 363-370. J. Rutkowski, Heuristic network partitioning algorithm using the concept of loop index, 1EEE Proc. 131-G(5) (1984) 203-208.
Tzvi Raz has B Sc. (1977Technion, Israel), MASc. (1979 - University of Toronto) and Ph D. (1982- University of Missouri) degrees, all in Industrial and Management Engineering. He has worked as an analyst in a military computing center (1979-1980) and as a management engineer in a VA hospital (1981-1982). He was an Associate Professor of Industrial Engineering at the University of Iowa (1982-1989) and at the University of Arizona (1988), and a Senior Lecturer at Ben Gurion University in Israel (1989-1990). He has taught and conducted research in the areas of computer and information systems, quality planning and control, production and operations management, and quantitative methods. Tzvi has authored or coauthored over 30 technical papers published in refereed journals, edited two books, and written a good number of book chapters, conference papers and technical publications of various types. He joined IBM in 1990 as an Advisory Development Analyst at the Resource Center for Software Excellence, Westlake Programming Laboratory, Roanoke, TX, where he works on operational problems in software development. Nancy Botten is an Industrial and Management Engineer at the IBM Westlake Programming Lab, where she is focusing on improving the software development process. Prior to this, she was employed by IBM Corporation in Rochester, MN, workin~ on productivity metrics and was Principal Researcher on a legal expert system development project. She completedher Ph.D. in Industrial and Management Engineering at the University of Iowa in Iowa City. She is currently a member of the Institute of Industrial Engineers, IE~EE and IEEE Computer Society, American Association of Artificial Intelligence, Dallas/Fort Worth Association for Software Engineering Excellence, Metroplex Institute for Neural Dynamics, and Alpha Pi Mu.
329
The knowledge base partitioning problem: Mathematical formulation and heuristic clustering Tzvi R a z and N a n c y A. B o t t e n IBM Corporation, Westlake Software Development Laboratory, Roanoke, TX 76299, USA Abstract Raz, T. and N.A. Botten, The knowledge base partitioning problem: Mathematical formulation and heuristic clustering, Data & Knowledge Engineering 8 (1992) 329-337. Allocation of production rules among several partitions in a knowledge base can shorten the compilation and execution of expert systems applications and facilitate their verification, validation ad maintenance. The problem of allocation of rules or groups of rules among partitions of limited size while minimizing the sum of inter-partition connections is formulated as a 0-1 integer programming problem with a quadratic objective function. Since this type of problems is NP-complete, a clustering algorithm based on the nearest neighbor heuristic is proposed.
Keywords. Knowledge base design; rule base programming; graph partitioning.
1. Introduction
The design of the knowledge base for an expert system involves acquiring the knowledge from one or possibly several domain experts, and coding it in a form compatible with the software environment of the system. Quite often, not all the knowledge is applicable to and required by each and every case for which the expert system might be used. It is also possible that substantial advantages in memory requirements and execution time may be achieved if the knowledge base is divided into a number of chunks or partitions, such that only one of them is loaded at any time. Since the partitions contain only a fraction of the knowledge in the system, each partition is less complex that the entire system. The reduction in complexity obtained by partitioning the knowledge base may facilitate its verification and validation in the design and coding stages, and its maintenance and updating throughout the life cycle of the system. Other potential benefits of knowledge partitioning include the possibilities of development in parallel and of reusability of coded knowledge. This discussion provides the motivation for allocating the knowledge contained in an expert system among a number of partitions. Of course, partitioning, like everything else, does not come free. Its costs derive from two main sources: additional overhead and switching among partitions. The cost of additional overhead is borne by the user, who has to remember what parts of the knowledge are kept in each partition and may have to switch partitions in the course of a session with the expert system. The overhead problem may be alleviated to some extent by good software design, especially in the user interface, but an expert system consisting of several knowledge bases will always be somewhat more complicated than one consisting of a single knowledge base. However, many users will gladly pay 0169-023X/92/$05.00 © 1992- Elsevier Science Publishers B.V. All rights reserved
330
7". R a z . N . A .
Botten
the price of additional overhead it it brings about a significant reduction in compilation and execution time. The cost of switching among partitions is directly related to the extent of reasoning connections among the rules in the different partitions of the knowledge base. A connection between two partitions exists if a line of reasoning requires knowledge stored in both of them in order to reach a conclusion useful to the user. Connections may be best explained in terms of production rules, which are among the most common forms of knowledge representation in expert systems. A production rule consists of premise part and a conclusion part. The premise contains a pattern of elements, which, if matched by the facts stored in the working memory of the expert system, causes the elements in the conclusion part to be executed. A connection between two partitions exists if one of the partitions contains a rule with a conclusion element that is a premise element in a rule that belongs to the other partition, or vice versa. A connection between partitions has a negative effect on the performance of the expert system. This is due to the fact that the partition connected to must be loaded (and, depending on the environment, may also need to be compiled), replacing the currently loaded partition while preserving the current contents of the working memory. This operation requires a relatively large amount of time, slowing down the response time of the expert system. Obviously, it is in our interest to partition the knowledge base in a way that minimizes the extent of switching among partitions. This paper deals with the problem of knowledge base partitioning in the context of rule-based expert systems. Following a review of the relevant literature, we present a mathematical formulation of the problem. Since the problem turns out to be NP-complete, we propose a heuristic clustering algorithm for knowledge base partitioning, and illustrate its application in an actual case study.
2. Review of the literature
Grouping of rules according to subject or use in order to assist designers, developers and users is advocated by many reference and text books on expert systems. For instance, Hayes-Roth et al. [10] and Martin and Oxman [16] mention the use of metarules (rules containing knowledge about the organization of the knowledge base) as a means for navigating among the various rule groups, and Cooper and Wogrin [6] devote a chapter of their text on rule-based programming to the implementation and control of rule clusters for improving the performance of OPS5 programs. The following papers addressing explicitly the issue of knowledge base partitioning are discussed in some detail. Jacob and Froscher [11] addressed the problem of production rule-base partitioning from the perspective of facilitating maintenance of the knowledge base. Their objective was to divide the rules into groups to reduce the amount of information that a knowledge engineer must understand in order to make changes, while limiting the flow of information among the groups. Rules are related to each other through elements the values of which they use or modify. The relationships can be described by a graph, with nodes representing rules and edges representing common elements. These authors presented two approaches to knowledge base partitioning. This first consists of dividing the rules into groups so that each group produces only one fact that is used by other groups. The second approach is a clustering algolJthm based on a measure of closeness between rules. The closeness between two rules is the weighted sum of their common elements. The weights reflect whether the element appears in both rules as a condition (0.33); as a conclusion (0.5) or as a condition in one rule and as a conclusion in the other (1.0). The closeness between a rule and a group of rules is the average of the closeness measures between the rule and each
The knowledge base partitioning problem
331
of the individual rules that belong to the group. Clustering proceeds by iteratively combining the closest pair of rules or rule groups and updating the closeness values. The authors reported that encouraging results were obtained in the analysis of several existing knowledge bases. Cheng and Fu [4] presented a clustering procedure for discovering the underlying concepts in a knowledge base consisting of production rules. The distance between two production rules is defined as the normalized number of elements not common to both. Rules groups ('stars') are created by bringing together the rules whose distance from the rule 'seed' is below a statistically determined threshold. The rule groups are replaced by representative rules, which contain the most frequent rule elements found in the group. The process is repeated until no additional changes result. At that point the elements of the representative rules constitute the underlying concepts of the knowledge base. The published literature containing quantitative techniques for knowledge base partitioning is relatively scarce. Several other references containing generic approaches or dealing with software partitioning in general are also available. The interested reader is directed to the comprehensive review prepared by Botten, Kusiak and Raz [1].
3. Problem statement
The knowledge base partitioning problem consists of finding the optimal assignment of production rules to a number of partitions. Typically the development of the knowledge base proceeds in a top-down manner by decomposing the domain into successively smaller topical areas, up to the point where the knowledge relevant to each of these areas can be expressed by a relatively small number of production rules. Since the rules within each such area are closely related, by grouping them together and assigning them to partitions as a group, we may reduce the need to switch among partitions in the course of a consultation session with the expert system. By committing to maintain the topical areas intact, the problem is redefined as the assignment of topical areas to partitions. Of course, there is no loss of generality in this approach, since if we wish to consider each rule in separate, then we define each topical area as containing exactly one rule. Two types of constraints are present. First, the assignment must be exhaustive and mutually exclusive, meaning that each and every topical area must be assigned to exactly one partition. Second, the size (in terms of the number of rules) of the partitions should not exceed a given value. The objective function to be optimized reflects the extent to which the partitions are independent of each other, as measured by the number of connections across partitions. A connection is defined as an instance of a rule with a premise element in one partition and a conclusion element in a different partition. We may assign different weights to the connections based on the number of elements involved, on the likelihood that the rule will be invoked in a session with the expert system, or on a subjective measure of the desirability of keeping the rules in the same partition. In any case, the objective of the problem is to find the assignment that minimizes the weighted sum of the connections across partitions.
4. Mathematical programming formulation
In this section we develop a mathematical programming formulation of the knowledge base partitioning problem. There are N areas to be assigned among M partitions. The following notation is used.
T. Raz, N . A . Botten
332
x~k = 1 if area i is assigned to partition k 0 otherwise a~j = weight of the connection between area i and area j A = adjacency matrix containing the weights of the connections between areas (a~j) bi = size of area i, in terms of number of rules or some other relevant measure B = maximum partition size. The smallest number of partitions that permits a feasible solution is the smallest integer not smaller than (Zi~ ~ bi)/B. However, this number does not assure the existence of a feasible solution. If the size of several areas is close to the partition size, then it may not be possible to fit more than one partition per area. For example, consider four areas with three rules each. Partition size cannot exceed four rules. Although in principle three partitions would contain the 12 rules in the system, in this case it is obvious that three would not suffice, and four are needed to obtain a feasible solution. To avoid this problem, one may formulate the problem with more partitions than the absolute minimal number, keeping in mind that the optimization procedure will ensure that unneeded partitions will remain empty. For any partition k the quantity Xik(1 -- Xjk ) is equal to 0 if and only if the two areas i a n d j belong to partition k, and is equal to 1 if area i belongs to partition k and area j belongs to a different partition. Consequently, the objective function may be formulated as: M
N
N
~ ~ Xik(l-
MIN ~ k=l
Xjk)aij
(1)
i=l j-I
Note that the triple summation yields twice the sum of the weights of the connections across partitions, since each connection is counted twice. Of course, this does not affect the solution. If, however, we prefer to deal directly with sum of the weights of the connections and reduce somewhat the number of calculations required, then we use the following objective function: M
N
i-1
~'~ ~ xig(1- xjk)aii
MIN ~ k=l
(2)
i=2 j=l
We may observe that the sum of all the weights of the connections among areas is constant. Consequently, minimization of the sum of the connections among partitions is equivalent to maximization of the sum of the connections within partitions. Thus we have an alternative objective function: M
MAX ~ k=l
N
~
i-1
~ XikXjka q
(3)
i-2 j=l
There are three sets of constraints: 1. The decision variables Xik must be 0 or 1: x~k E {0, 1)
for all areas i = 1 , . . . , N and all partitions k = 1 . . . . .
M
(4)
2. Each area must belong to exactly one partition: M
x~=l k=l
for all a r e a s i = l , . . . , N
(5)
The knowledge base partitioning problem
333
3. The partition sizes should not exceed the limit B: N
Xikb i <<-B
for all partitions k = 1. . . . . M
(6)
i=1
The three sets of constraints in equations (4), (5) and (6) are applicable regardless of which formulation (1), (2) or (3) we choose for the objective function. In any case we have an integer quadratic programming problem with M x N 0-1 decision variables, N equality constraints and M inequality constraints. In a series of papers, Picard and Ratliff [18-20] established the equivalence between 0-1 integer programming with a quadratic objective function and the graph partitioning problem. This problem has received a fair amount of attention in the literature, which we review briefly next. Gorinshteyn [9] developed a branch and bound method to the graph partitioning problem when all the nodes are of size 1. Kernighan and Lin [13] developed a heuristic for minimizing the sum of the weights on the cut-sets when all the nodes are of equal size. Later on, Kernighan [12] developed an exact optimization algorithm for a variation of that problem with the additional constraint that the assignment of nodes to subsets must preserve a linear order of the nodes. Christofides and Brooker [5] proposed a tree search method for the optimal partitioning of a weighted graph into two parts, using the concept of the longest spanning tree to direct the search, with bounds calculated with a maximum flow algorithm. The version of Minoux and Pinson [17] of the graph partitioning problem has a constraint on the number of subsets rather than on the subset size. The algorithm of Duncan and Scott [7] for partitioning a weighted graph with equal sized nodes in order to minimize the interset weights is based on a probabilistic branch and bound approach. Rutkowski [21] approached the graph partitioning problem by tearing nodes instead of disconnecting edges. His solution is based on heuristics for permutating the rows and columns of the fundamental circuit matrix corresponding to a tree in the graph. Lawler, Levitt and Turner [15] used graph partitioning to solve the problem of allocation of electronic gates to modules (subsets) of limited size in order to minimize transmission delay for the entire network. Their approach consisted of a heuristic clustering algorithm designed to minimize the length of the longest path in a directed graph, with the distance between gates being defined as the number of modules that the signal had to travel through. Breuer [3] also studied the electronic circuit partitioning problem and developed three heuristic placement algorithms based on finding the minimal cut in the circuit graph for minimizing transmission path length. Krishnamurthy [14] expanded the work of Kernighan and Lin [13] as it applies to VLSI network design and developed improved heuristics for selecting the pair of nodes to interchange. None of the variations of the graph partitioning problem solved is directly applicable to the knowledge partitioning problem, which has the following characteristics: the graph is not directed and its nodes are not ordered; both nodes and edges have weights; the subset size is constrained from above; and the objective function is the sum of the weights of the cut-set edges. In fact, this particular type of problem appears in the list of NP-complete problems compiled by Garey and Johnson [8]. Although small instances of the problem may be solved with the assistance of specialized software, in general the computational time and resources required are quite large and increase exponentially with the number of topical areas and partitions. This situation justifies the use of heuristics for solving the knowledge base partitioning problem.
T. Raz, N.A. Botten
334
5. Heuristic clustering In the following section we present a clustering approach based on the nearest neighbor, or greedy, heuristic: we attempt to assign to the same partition the areas that are most closely connected. The areas are combined into clusters in descending order of their connection weights air This continues until the number of clusters equals the desired number of partitions, while skipping over those combinations that result in cluster size larger than the given limit on the partition size. This algorithm operates on the adjacency matrix A, which contains the weights of the connections between areas (a~j). At each iteration, some of the values in the matrix A are changed to keep track of the areas that were considered for combination. The algorithm is summarized next.
Initialization: 1. Each area is a cluster 2. a , ~ 0 for all i (ignore the weight of the connections within areas) Do While Not all aij = 0 Step 1. Find the two clusters i' and j' (i' < j ' ) with the largest connection weight between them (ties are decided arbitrarily) ai, r = MAX aq ij~[1, N]
a t / , , a j, i, ~ O (remove connection from further consideration)
Step 2. Check for feasibility of combination: If b i, + b r <~ B (combination feasible) then proceed to Step 3 Otherwise, go to Step 1
Step 3. Combine cluster i' and cluster j': For all k: ark ~ ai,~ + aj, k
a/,k ~ 0 aki, ~ aki, + akj,
akj, ~ 0
(combine row of ]' with row of i') (remove from further consideration the combined row) (combine column of j' with column of i') (remove from further consideration the combined column)
Go to Step 1
E n d Do While The algorithm can have at most N iterations. The dominant calculation in each iteration consists of finding the maximum of at most N 2 aij values, which requires N2-1 comparisons. By taking into account the fact that the A matrix is symmetric with respect to its diagonal, the number of comparisons may be reduced by half. Additional reductions in the number of comparisons may be obtained by reducing the order of A as the iterations progress. However, these will not bring a reduction in the order of magnitude. Overall, the computational complexity of the algorithm is of the order of N * N 2, or O(Na). The computational effort required to obtain the adjacency matrix A should also be considered.
The knowledge base partitioning problem
335
Defining the weights of the connections a, as a function of the number of connections between area i and area j, the values of a 0 are obtained by comparing the premise part of every rule to the conclusion part of every rule not in the same area. The resulting computational complexity is of order O(K2), where K is the total number of rules in the knowledge base. We note that the complexities of both the clustering algorithm and the setting up of matrix A are low-order polynomials.
6. Example The data for this example is taken from a project that involved the design of a knowledge base with 178 rules divided into 15 topical areas. The maximum allowed partition size was set equal to 50 rules. Full details of the case appear in Botten and Raz (1990). The entries a 0. of the adjacency matrix A represent the number of rules in area i that either have a premise (IF) element the value of which is affected by an action (THEN) element of a rule in area j, or have an action element that affects the value of a premise element of a rule in area j. Based on this definition, aij = a~ and A is symmetric. Since internal connections within areas are ignored, a , = 0 for all i. The adjacency matrix and the number of rules in each area appear in Fig. 1. At the beginning each area is a separate cluster. We identify the two clusters with the largest connection weight, which are cluster 3 and 4, with a34 -- a43 = 5. In the first iteration, these two clusters are combined, with the result that cluster 3 now contains areas 3 and 4, and all the entries involving area 4 have been set equal to 0. Figure 2 shows the resulting adjacency matrix and cluster sizes, with the values affected by the iteration in bold face. Now the two clusters with the largest connection weight between them (8) are cluster 3, which includes areas 3 and 4, and cluster 10, which so far includes area 10 only, and the two are combined. The results of the algorithm at each iteration are summarized in the dendogram in Fig. 3, which also shows the value of aii that was used for the combination and the size of the newly created cluster. Since the smallest number of partitions is 4, there were 1 5 - 4 = 11 iterations. The four final clusters, each corresponding to a partition of the knowledge base, are: {3, 4, 7, 9, 10}; {1, 2, 6, 12, 13}; {5, 8, 14}; and {11, 15}. The sum of the weights of the connections among the partitions was 31, which turned out to be equal to the optimal value found by enumeration. area
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
,1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 size
0 1 1 1 0 1 2 1 4 1 0 3 3 0 0 11
1 0 1 0 0 0 2 0 4 3 0 0 0 0 0 10
1 1 0 1 0 1 1 1 1 4 2 0 0 0 0 5
1 3 5 3 0 0 3 0 3 4 0 0 0 0 0 8
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 9
1 0 1 0 0 0 0 0 0 0 0 0 1 0 0 4
2 2 1 2 0 0 0 2 4 3 0 0 0 0 0 11
1 0 1 0 0 0 2 0 0 0 0 0 1 1 0 16
4 4 1 4 0 0 4 0 0 3 0 0 1 0 0 5
1 3 4 3 0 0 3 0 3 0 0 0 0 0 0 18
0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 17
3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8
3 0 0 0 0 1 0 1 1 0 0 0 0 0 0 15
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 22
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19
Fig. 1. A d j a c e n c y matrix and cluster sizes before iteration 1.
336
T. Raz, N . A . Batten
area
5
6
7
8
9
10
11
12
13
14
15
1 3 8
0 0 2
3 0 0
3 0 0
0 0 0
0 0 0
0
1
0
1
2
0
2 3
1 2
0 4
4 0
0 0
0 0 0
1 0 I
2 2 4
I 0 I
4 4 4
4
0
0
0
0
0
0
0
0
0
0
0
0
0
0
5 6 7 8 9 10 11 12 13 14 15 size
0 1 2 1 4 1 0 3 3 0 0 11
0 0 2 0 4 3 0 0 0 0 0 10
0 1 4 1 4 8 2 0 0 0 0 13
0 0 0 0 0 0 0 0 0 0 0 0
0
0
0
0
0
0
0
0
0
0
0 0 0 0 0 0 0 0 0 0 9
0 0 0 0 0 0 0 1 0 0 4
0 0 2 4 3 0 0 0 0 0 11
0 0 2 0 0 0 0 0 1 1 0 16
0 4 0 0 3 0 0 1 0 0 5
0 3 0 3 0 0 0 0 0 0 18
0 0 0 0 0 0 0 0 0 0 17
0 0 0 0 0 0 0 0 0 0 8
1 0 1 1 0 0 0 0 0 0 15
0 0 1 0 0 0 0 0 0 0 22
0 0 0 0 0 0 0 0 0 0 19
Fig. 2. Adjacency matrix and cluster sizes after iteration 1. Areo 3 4 I0 7 9 I 12 15 6 2 8' 14 5 11 15 Iteration Largest a~j
I]
I..............................
]- . . . . . . . . . . . . . . . . . . . . . .
t.....................
I I1 I2I3I4I5I 6 5 8 7 1 1 3 3 2
t 9t 10Jt11 12t 13t 14It 15
8
1
1
0
0
New Cluster Size 13 31 42 47 19 34 38 48 38 47 36
Fig. 3. Dendogram of the iterations of the clustering algorithm.
7.
Concluding
remarks
T h e d e v e l o p m e n t , use a n d m a i n t e n a n c e of a k n o w l e d g e base m a y be facilitated by its d i v i s i o n i n t o several p a r t i t i o n s . I n such a case, it is i m p o r t a n t to assign the rules or g r o u p s of rules to p a r t i t i o n s in a m a n n e r that m i n i m i z e s the n u m b e r of i n t e r p a r t i t i o n c o n n e c t i o n s . This t y p e of p r o b l e m is e q u i v a l e n t to the g r a p h p a r t i t i o n i n g p r o b l e m w h e n n o d e s a n d arcs have weights, t h e r e is a n u p p e r limit o n the p a r t i t i o n size a n d the o b j e c t i v e is to m i n i m i z e the s u m of t h e weights of the cut sets. I n this p a p e r we p r o p o s e d a clustering a l g o r i t h m b a s e d o n the n e a r e s t n e i g h b o r heuristic. I n o u r e x a m p l e , which is t a k e n f r o m a n actual e x p e r t system d e v e l o p m e n t p r o j e c t , the heuristic a l g o r i t h m yielded a n o p t i m a l solution. O b v i o u s l y , m o r e e x p e r i m e n t a t i o n is r e q u i r e d if o n e is i n t e r e s t e d in the p e r f o r m a n c e of the heuristic u n d e r d i f f e r e n t c o n d i t i o n s . H o w e v e r , we b e l i e v e that its low c o m p u t a t i o n a l c o m p l e x i t y , O ( N 3 ) ,
The knowledge base partitioning problem
337
and its intuitive logic make it an attractive alternative to solving the exact mathematical
formulation of the knowledge base partitioning problem (0-1 integer-programming with a quadratic objective function), which happens to be NP-complete. References [1] N. Botten, A. Kusiak and T. Raz, Knowledge bases: Integration, verification and partitioning, Europ. J. Operat. Res. 42 (1989) 117-128. [2] N. Botten and T. Raz, Knowledge base development for an expert system for civil rights investigation, Expert Systems 7 (1990) 224-235. [3] M.A. Breuer, Min-cut placement, J. Design Automation and Fault-tolerant Comput. 1 (1977) 343 -362. [4] Y. Cheng and K. Fu, Conceptual clustering in knowledge organization, IEEE Trans. Pattern Analysis Machine Intelligence. PAMI-7 (1985) 592-598. [5] N. Christofides and P, Brooker, The optimal partitioning of graphs, SIAM J. Applied Mathemat. 30(1) (1976) 55-69. [6] T.A. Cooper, and N. Wogrin, Rule-Based Programming with OPS5 (Morgan Kaufmann, San Mateo, CA, 1988). [7] J. Duncan and L.W. Scott, A branch-and-bound algorithm for pagination, Operat. Res. 23(2) (1975) 240-259. [8] M.R. Garey and D.S. Johnson, Computers and Intractability- A Guide to the Theory of NPCompleteness (W.H. Freeman, New York, 1979) [9] L.L. Gorinshteyn, The partitioning of graphs, Engrg. Cybernet. 1 (1969) 76-82. [10] F. Hayes-Roth, D.A. Waterman and D.B. Lenat, eds., Building Expert Systems (Addison-Wesley, Reading, MA, 1983). [11] R. Jacob and J. Froscher, Developing a software engineering methodology for rule-based systems 1985 Conf. on Intelligent Systems and Machines, Oakland University (1985) 179-185. [12] B.W. Kernighan, Optimal sequential partitions of graphs, J. Assoc. Comput. Machinery 18(1) (1971) 34-40. [13] B.W. Kernighan and S. Lin, An efficient heuristic procedure for partitioning graphs, Bell Syst. Tech. J. 49(2) (1970) 291-307. [14] B. Krishamurthy, An improved min-cut algorithm for partitioning VLSI networks, IEEE Trans. Comput. C-33(5) (1984) 438-446. [15] E.L. Lawler, K.N. Levitt and J. Turner, Module clustering to minimize delay in digital networks, IEEE Trans. Comput. C-18(1) (1969) 47-57. [16] J. Martin and S.W. Oxman, Building Expert Systems (Prentice-Hall, Englewood Cliffs, N J, 1988). [17] M. Minoux and E. Pinson, Lower bounds to the graph partitioning problem through generalized linear programming and network flows. RAIRO
[18]
[19] [20]
[21]
Recherche Operationelle/ Operat. R e s . 21(4) (1987) 349-364. J.C. Picard and H.D. Ratliff, A graph-theoretic equivalence for integer programming, Operat. Res. 21(1) (1973) 261-269. J.C. Picard and H.D. Ratliff, Minimum cuts and related problems, Networks 5 (1975) 357-370. J.C. Picard and H.D. Ratliff, A cut approach to a class of quadratic integer programming problems, Networks 10 (1980) 363-370. J. Rutkowski, Heuristic network partitioning algorithm using the concept of loop index, 1EEE Proc. 131-G(5) (1984) 203-208.
Tzvi Raz has B Sc. (1977Technion, Israel), MASc. (1979 - University of Toronto) and Ph D. (1982- University of Missouri) degrees, all in Industrial and Management Engineering. He has worked as an analyst in a military computing center (1979-1980) and as a management engineer in a VA hospital (1981-1982). He was an Associate Professor of Industrial Engineering at the University of Iowa (1982-1989) and at the University of Arizona (1988), and a Senior Lecturer at Ben Gurion University in Israel (1989-1990). He has taught and conducted research in the areas of computer and information systems, quality planning and control, production and operations management, and quantitative methods. Tzvi has authored or coauthored over 30 technical papers published in refereed journals, edited two books, and written a good number of book chapters, conference papers and technical publications of various types. He joined IBM in 1990 as an Advisory Development Analyst at the Resource Center for Software Excellence, Westlake Programming Laboratory, Roanoke, TX, where he works on operational problems in software development. Nancy Botten is an Industrial and Management Engineer at the IBM Westlake Programming Lab, where she is focusing on improving the software development process. Prior to this, she was employed by IBM Corporation in Rochester, MN, workin~ on productivity metrics and was Principal Researcher on a legal expert system development project. She completedher Ph.D. in Industrial and Management Engineering at the University of Iowa in Iowa City. She is currently a member of the Institute of Industrial Engineers, IE~EE and IEEE Computer Society, American Association of Artificial Intelligence, Dallas/Fort Worth Association for Software Engineering Excellence, Metroplex Institute for Neural Dynamics, and Alpha Pi Mu.