- Email: [email protected]
Parallel architectures for AI semantic network processing
Parallel architectures for AI semantic network processing Jos6 G Delgado-Frias and Will R Moore
Artificial intelligence (AI) applications are growing in several fields, and in many such applications knowledge bases must be manipulated. This activity is usually performed by an external agent such as a central processor, but often this cannot supply the speed required. Knowledge-oriented architectures provide an efficient execution of knowledge manipulations. This paper provides an introduction to a particular subset of knowledgeoriented architectures, the semantic network approach, which is one of the most commonly used methods of representing and manipulating knowledge in the A I field. A brief overview of the semantic network components is presented in order to provide a background to the topic. The purpose of this paper is to review the proposed, implemented and/or simulated architectures for semantic network processing, and to discuss the capabilities and limitations of such architectures. Keywords: semantic networks, parallel computers, artificial intelligence, knowledge-oriented systems
Knowledge representation and manipulation is required in many AI algorithms. Processing knowledge representations requires a large computational power that may not be delivered by conventional uni-processor architectures at a reasonable speed. Special purpose architectures are needed to manipulate such knowledge representations. The main goal of knowledge-oriented systems is to provide efficient execution of models for the representation and manipulation of knowledge 1. Knowledgeoriented architectures are divided into two approaches: symbolic manipulation and connectionism 2. Rule-based machines, such as DADO 3 and FAIM-14 are examples of symbolic manipulation architectures. In this approach, knowledge is represented as a collection of rules which are manipulated in order to make inferences. The connectionism approach is a class of massively parallel architectures where knowledge is distributed over the whole system s. A massively parallel architecture VLSI ResearchGroup, Department of Engineering Science,University of Oxford, Parks Road, OxfordOXI 3PJ, UK
manipulates knowledge by trying out different connections in the network until a set of solutions is found. The Connection Machine (CM) 6 and NETL 7 are examples of this approach. This paper deals with knowledgeoriented massively parallel connectionist systems, and in particular with semantic networks.
SEMANTIC NETWORKS Quillian s introduced the idea of a semantic network, although several proposals and changes have been made since (Brachman 9 provides an overview of these proposals). The basic principle of a semantic network is simple: each node of the interpreted graph represents a concept, and the links between them represent relationships between concepts ~°. Figure 1 shows such a semantic network, and the more important issues of semantic networks are briefly described below.
Hierarchical representation Knowledge in a semantic network is represented in a hierarchical form. The knowledge of whether or not an item belongs to a set is important for answering questions and retrieving facts 11. A method of providing a hierarchical representation is to use the arcs (or links) to classify the concepts represented by various nodes 1H3. An example of this method is shown in Figure 1; the
is-a
is-a
~
Taste
is-a
I
© (o.-,,)/
I
")
(Co~ou~.--~--~
S-a
~s-a
Figure I. A semantic network
0950-7051/88/050259-07 $03.00 © 1988 Butterworth & Co (Publishers)Ltd Vol I N o 5 December 1988
259
taxonomy is provided by means of arcs labelled 'IS-A'. For instance, in Figure 1 the arc between ELEPHANT and M A M M A L represents the knowledge that ELEPHANT is a subset of MAMMAL.
Inheritance As soon as humans beings identify something, many assumptions are made; for example, that birds fly, whales live in water, etc. A similar process can be done in semantic networks by means of inheritance. In a hierarchical representation, nodes at the lower levels inherit all the properties of nodes above them. The hierarchy determines the way that inheritance is passed; properties of a node do not affect the properties of nodes at higher levels. In Figure 1, for example, Clyde is described as an elephant; however, there is no direct link that determines Clyde's colour. By using inheritance, the colour of Clyde can be obtained from the elephant description. Inheritance from another semantic network tree must be disallowed in order to avoid contradictions; for instance, no node below PLANT can inherit anything that is below ANIMAL.
Implicit and explicit knowledge A human mind can store a great quantity and variety of knowledge, and can access whatever knowledge it needs very quickly and flexibly 14. For example, if it is stated that Clyde is an elephant, a human being can tell a great deal more about Clyde than what was stated explicitly. It is possible to infer (with a high degree of certainty) the colour, the number of legs and the size of Clyde. It is impossible to believe that all the knowledge about Clyde is stored in explicit form. The most feasible alternative is that knowledge is stored in an implicit form. A semantic network must be able to handle implicit knowledge since it would be almost impossible to store all facts explicitly. Figure 1, for example, shows how implicit knowledge is set in a semantic network. It can be seen that Clyde is an elephant, a mammal, an animal and a physical object by using implicit knowledge. Also, it can be observed that Clyde is a male and a veteran by using explicit knowledge representation. IS-A link Much representation of the world is concerned with the conceptual relations expressed in English sentences such as Peter is a bachelor and A dog is a domesticated mammal. The easiest way to get those statements into a semantic network scheme is to have a link that represents the IS-A part of such sentences I ~ IS-A connections form a hierarchy which makes semantic networks an efficient storage scheme. The IS-A link also allows the notion of inheritance of properties. There are a number of uses for an IS-A link 15, among them: • Subset/superset: one node is a superset of another node, e.g. the elephant set is a subset of mammals. • Generalization/specialization: some properties of nodes may be true for a node below it. This is usually conditional to definitions on the higher level node, providing there is no contradictory information. • Set membership: nodes that represent individuals are 260
set members of a higher level node, e.g. Clyde is a member of the ELEPHANT set. • Conceptual containment: a description includes another, e.g to be a triangle is to be a polygon with three sides; here a predicate is used in defining another. SEMANTIC NETWORK ARCHITECTURES Knowledge representations have been implemented on single processor architectures. This approach, however, presents a serious drawback: processing time is enormous due to the large knowledge bases and no a priori searching path. A simple query may lead to a search into many nodes in the knowledge tree. Some search-guiding heuristic algorithms may help to reduce the amount of searching. Such algorithms, however, are problem-domain dependent and carefully handcrafted to handle each particular set of problems 1.. On a serial machine this process takes a time proportional to the number of the nodes that have to be processed. With the advent of Very Large Scale Integration (VLSI), the possibilities of Ultra Large Scale Integration (ULSI), and the possible advantages of Wafer Scale Integration (WSI), a cost effective parallel architecture which can handle semantic networks may be achievable. Advantages of such an architecture include: • Time is proportional to the number of nodes in the branch that provides the answer. • Several alternatives are searched in parallel: if a query has more than one correct answer, they can be obtained with minimum extra effort. • Problem-domain independence: node and link processing is executed in parallel so there is no need for problem-domain heuristic algorithms. • Each node and link executes a small number of simple instructions. Several systems for semantic networks are described here, most of which have been implemented in hardware or simulated, though some are only proposals. NETL The NETL system was proposed by Fahlman 7''0'17. The NETL architecture consists of very simple hardware elements called 'node units'; relationships among concepts are represented by hardware elements called 'link units'. A serial controller is used to send simple instructions to node and link units by means of a shared bus. Figure 2 shows the basic NETL hardware. Each node has its own serial number, a few bits of information about the concept type that it represents, and a few bits to mark the nodes. Links also have a few bits for information about relationship type. Two wires come out of the link unit, wire A being connected to the object being described, and wire B going to the node that represents the class where the object has been classified. Nodes and links are able to perform simple Boolean operations. In a more recent version of NETL, a circuit switching network called 'hash net' is used to establish communication between a node and a link 16. Hashnet is a permutation network that is able to estabKnowledge-Based Systems
]
IIIIIIIIII
0Elephant, node unit
.Q
~.~
[
Hasnet
qs-a' link unit
I I I! I I I I II I
I
'Clyde' node unit
j
Serial control computer
Figure 2. NETL system hardware Thing
1 i
Router
1
Memory
Memory
Figure 4. VLSI module connection machine hardware Clyde
Figure 3. Marker propagation scheme for NETL (all solid arcs are 'is-a" links) lish one-to-one connections which are able to operate simultaneously. NETL can perform searches and inferences by means of a marker propagation scheme. In order to understand how the marker propagation scheme works, consider the example shown in Figure 3. Suppose that the colour of Clyde is needed. To start with, the controller sets the CLYDE node to marker one. Then all nodes above in the hierarchy (that are connected with IS-A links) are told to set a copy of marker one. In the next step the controller sends a command to all COLOUR-OF links with marker one and their A wire, telling them to set marker two in the node tied to their B wire. Finally, all nodes with marker two place their contents on the bus. The answer in the example is GREY. Fahlman found that the marker passing scheme may not be able to handle some recognition problems (such as vision, speech understanding, medical diagnosis, etc.) s. Value passing may be a better solution for this type of problem. Vol 1 No 5 December 1988
Connection Machine The Connection Machine (CM) system was proposed by Hillis6'1a (CM) and the prototype has been built by Thinking Machines Corporation (TMC). The CM can perform most operations at a rate in excess of 1000 MIPS 19. The TMC version of the full CM is intended to be a 1024 x 1024 PE array that is linked into a 14-cube. The system that has been built has 64K (256 x 256) synchronized 1-bit processors, each with a 4 Kbit memory. Figure 4 shows the architecture of the CM. The network of the CM was the prime effort of the machine design 6. The CM prototype has a 12-cube network with a packet switching scheme. Each VLSI chip has 16 processing elements and a router processor which is in charge of communicating between that PE cluster and the network. The CM is programmed in extensions of Common Lisp and C. The original purpose of the CM design was to manipulate knowledge stored in semantic networks concurrently xs. Each node and link was mapped onto a processing element and data was transferred by means of a message passing scheme. The resulting architecture, 261
Host computer
I Control logic
Memory Global bus
t
m
Cell
Cell
]
Instruction reg.
I
Mask reg.
I
h
]_]_" I
al bus
,I
Figure 5. Semantic network array processor (SNAP) architecture however, turned out to be a quite general purpose machine. This architecture executes data parallel algorithmsZ°: document retrieval from a large database 21, memory-based reasoning and natural language processing 19. In document retrieval, for example, from a database of 160 Mbyte the CM can operate at an effective execution speed of 6 000 MIPS zz. Another version of the CM is being developed at the MIT Artificial Intelligence Laboratory23. In this version the reconfiguring network is an NlogN network rather than a N-cube. This network needs logN stages to send data from one node to another.
Semantic Network Array Processor The Semantic Network Array Processor (SNAP) architecture is currently under study at the University of Southern California24. The architecture consists of a square array of identical processing dements which are connected in a Nearest Neighbour Interconnection (NNI) scheme for local communication, and in a column scheme for global communication (see Figure 5). The SNAP architecture is operated by a single controller which also acts as an interface between the array and a host computer. Each cell can be microprogrammed if it is required; this feature allows cells to operate independently. Each cell (or processing element) as shown in Figure 6, has three units: a Content-Addressable Memory (CAM), a Processing Unit (PU) and a Communication Unit (CU). The CAM has two modules: cell memory, which is the node local memory, and pointer memory, which is further subdivided to hold relation names and the addresses to which they relate. The PU controls both processing in the CAM and the operation of the CU. The PU has a reduced set of primitive (micro) instructions: AND, OR, NOT, SET, TEST, RESET, 262
MASK, MATCH, CAR and CDR. Data transfer between cells is executed by the CU. SNAP is capable of executing AI inference algorithms (i.e. deducing new facts from existing ones), since these algorithms are based on extensive searches and on a set of inference rules. The regularity of the SNAP array favours VLSI implementation. The performance of SNAP architecture has been observed by means of a simulation program which was written in UCI Lisp. SNAP simulation has revealed that little is gained in speed when eight nearest neighbour connections are allowed rather than four. This result is probably due to a low branching factor - - many semantic networks have a small number of links at each node. Also, it was observed that a slight improvement in processing time is achieved by a spiral allocation scheme. The SNAP system has been used to manipulate semantic networks in several applications, one of them being in high level computer vision25'26. Edge interpretation, and scene labelling are the main tasks in this application.
Parallel Network Wave Machine The Parallel Network Wave Machine (PNWM) system has been proposed by Sapaty and Kocis at the Slovak Academy of Sciences2L The main objective of the PNWM system is to obtain high performance on a parallel semantic network architecture by means of a distributed controller. Part of the control is sent to each node along with the data. A language called Wave was developed to achieve this control scheme 2s. The principle of this scheme is to propagate the formulae of this language in a dataflow mode, i.e. the program is sent along with data. The proposed PNWM architecture, shown in Figure 7, has five major blocks: a Language Processor (LP), a Planning and Control Processor (PCP), a Specialized Processor (SP), a Parallel Network Environment (PNE), and a Mass Memory (MM). The LP transforms user requests into internal wave language form. The PCP makes the initial activation of the PNE which is the main component of the system. The MM works as an extension of PE's local memory. The semantic networks are processed by the PNE block. The proposed PNE organization, shown in Figure 8, consists of a torus-like two-dimensional array with a four neighbour interconnection scheme. Each PE is also connected to two common buses which allow direct communication of any two processors in the same row or column. The PNWM is planned to be engaged in information systems, expert systems, CAD and modelling.
Dataflow approach The use of dataflow architectures29 for semantic network processing has been proposed by Bic30. In this approach, the semantic network is considered to be an interpreted dataflow graph. Each node is able to accept, process, and emit value tokens which are transferred asynchronously in the network. The semantic network is a dataflow graph as shown in Figure 9(a), where tl and t2 are constants in the nodes, and p is the label of the link (or arc) which is a constant. A query must also be a graph where at least one of the nodes or links corresponds to a constant while the others are variables. Figure 9(b) shows a query graph: T1, T2 and p can be either variables or constants. Knowledge-Based Systems
(i-1 ,j)
1
Cell (i,j)
IIO b u f f e r Global data bus
Global instruction bus
tl
CU
CAM
Busy full wait
Cell memory
Pocket r e g i s t e r
Prev pointer memory
t
I I n p u t queue
(i,j-1)
:
'Next' pointer memory
!6 general purpose flags
¢. re. :3 J2
,11,
O
t. r
iPV
:3 JD
-I~ ( i . j + l )
O
General purpose r e g i s t e r s Routing controller
Instruction memory
Control logic
I/O buffer
(i+I ,j)
Figure 6. SNAP processing element ~r user
r LP
PCP
SP
PNE
I
T
MM
b
Figure 7. Parallel network wave machine ( P N W M ) architecture The dataflow graph, representing either a semantic network or a query, must not have cycles, since those are difficult to detect due to the asynchronous scheme of dataflow systems. In order to have non-cyclic forms, the template must be transformed into a tree structure or a linear chain. Processing a linear template, such as the one shown in Figure 10, is performed thus: the template is sent as a token to all nodes in the semantic net that match Vol 1 N o 5 December 1988
Figure 8. P N W M system PNE module tI
t2
TI
T2
O
O
O
O
a
b
Figure 9. Dataflow ( a) semantic network, ( b) query graph 263
T°
Pi Ti+1 Pi+1
I
0
0
Ti+l
0
Pi+1
Pn-1 ---
T
n
0
Tn
.._ Pn-10
Figure 10. Linear graph template the leftmost node (T, in the template); nodes which receive this token will replicate it along all edges that match the first template edge Pl; these nodes then compare the second template/'2, and the operation continues until the last template is compared; all the values of the variables can then be obtained. All possible matches are found without the need for centralized control (a data-driven model of semantic network computations is presented in Reference 31).
SUMMARY There are two major approaches to parallel semantic network architectures: synchronous and asynchronous. The synchronous scheme is essentially a Single Instruction Multiple Data (SIMD) machine style. A central controller sends instructions to all PEs; some PEs may or may not execute each instruction, depending on their internal status. NETL, the CM, and SNAP are examples of this approach. Advantages of this approach include: simple processing elements, since almost no controller hardware is implemented in the PE; a simple communication scheme, because the controller only allows a few paths at any given time, thus avoiding collisions and higher PE density; and, since cells are small and regular, more of them may be put into a VLSI/WSI package. However, there may be some problems in a SIMD machine, such as some PEs possibly being idle for long periods of time when they are not in the tree that is being searched, and the central controller may be the bottleneck of the system, as nothing can be done until the controller indicates so. The asynchronous approach results in a Multi-Instruction Multi-Data (MIMD) streams machine style. A distributed controller, which is based on the dataflow philosophy, moves data and instructions around the PEs in the machine. The PNWM and Dataflow are examples of this approach. The MIMD architectures for semantic networks may provide the potential for high parallelism, since self-contained tokens may be processed simultaneously by several PEs. Problems in these architectures include the message passing overhead perhaps being quite large, complex processing elements are needed to process data and control flow, and the software to run these machines is complex. The 2-D array organization seems to be the preferred architecture for semantic network systems. Such an array has been used in the CM, SNAP and PNWM systems. The 2-D array advantages are a regular network, a well studied architecture, and a simple but powerful communication scheme for semantic network applications. Such advantages favour VLSI and WSI implemen264
Table 1. Semantic network processing systems
System
Developed at
Features
NETL
MIT
Simple PEs, single controller, node and link execute instructions
CM
Thinking Machines
64K PEs, 12-cube network, single controller
SNAP
USC
2-D array with global comms, scheme, nodes may be microprogrammed
PNWM
Slovak
2-D array with distributed system control
Dataflow UC, Irvine
Asynchronous scheme based on dataflow graphs
tations. Semantic networks with high node fanout may not be efficiently processed in a 2-D array; other types of networks may be useful, for example the n-cube and the cosmic cube. Simple processing elements are usually preferred over more complex ones, for instance, the PEs in NETL and CM architectures. This preference may be due to the simplicity of the semantic network instructions, and also to the need for having high PE density in the system. However, there are other approaches where the PEs are more powerful and execute complex instructions, e.g. the PEs in systems such as the SNAP, Dataflow and PNWM. The most common applications where semantic network engines have been used are knowledge retrieval and inferencing. Table 1 provides a summary of the semantic network architectures that have been described in this paper. CONCLUSION Artificial intelligence processing features 24'32 that influence the design of semantic network systems are: • Large problem size: the amount of information and the degree of parallelism are very large, therefore a great deal of computational power is required. • Memory access is intense: to speed up computation and avoid bottlenecks it is necessary to process data inside the memory. • Pattern retrieval operations are frequently used: parallel search may be the most promising approach to this operation. • Non-deterministic algorithms: there is no unique path from initial state to the goal. Parallel implementation of semantic nets is required in order to reduce processing time. The connectionism approach may be the most viable alternative to parallel semantic network processing. Several attempts towards a parallel semantic network have been presented in this paper. Most of them are based on simple processing elements connected, basically, in a two-dimensional array. A very large number of PEs are required in order to implement a semantic Knowledge-Based Systems
network system. A machine with so many active components cannot reasonably be expected to operate reliably without some form of fault-tolerances . However, few of these approaches have considered fault-tolerant schemes. Wafer-Scale Integration (WSI) technology provides some advantages (such as fast intermodule communication, higher reliability and lower pin-out count) which may benefit semantic network processing systems. WSI requires fault-tolerant schemes in order to overcome defects on the wafer, but a large system such as a knowledge-oriented architecture needs fault-tolerance anyway. Having a semantic network machine implemented in WSI may result in a faster, more reliable and possible cheaper system than having such a system in discrete VLSI chips. A WSI semantic network architecture with custom fault-tolerant schemes may have a great impact on the artificial intelligence field, in particular in areas where intense knowledge processing is needed. REFERENCES 1 Hwang, K, Gosh, J and Chowkwunyun, R 'Computer architectures for artificial intelligence' Computer Vol 20 No 1 (January 1987) pp 19-27 2 Treleaven, P C, Refenes, A N, Lees, K J and MeCabe, S C 'Computer architectures for artificial intelligence' Technical Report 119 (UCL-CS-TR 119) Department of Computer Science, University College London, UK (March 1986) 3 Stolfo, S J 'Initial performance of the DADO2 prototype' Computer Vol 20 No 1 (January 1987) pp 75-83 4 Anderson, J M, Coates, W S, Davis, A L, Hon, R W, Robinson, I N, Robinson, S V and Stevens, K S 'The architecture of FAIM-I' Computer Vol 20 No 1 (January 1987) pp 55-65 5 Fahlman,S E and Hinton, G E 'Connectionist architectures for artificial intelligence' Computer Vol 20 No 1 (January 1987)pp 100-109 6 Hillis, W D The Connection Machine MIT Press, USA (1985) 7 Fahiman, S E NETL: A System for Representing and Using Real-Worm Knowledge MIT Press, USA (1979) 8 Quillian, R M 'Semantic memory' in Minsky, M (ed) Semantic Information Processing MIT Press, USA (1968) 9 Brachmun, R J 'On the epistemological status of semantic networks' in Findler, N V (ed) Associative Networks Academic Press, USA (1979) pp 3-50 10 Woods, W A 'What's in a link: foundations for semantic networks' in Braehman, R J and Levesqne, H J (eds) Knowledge Representation MorganKaufmann, USA (1985) 11 Hendrix, G G 'Encoding knowledge in partitioned networks' in Findler, N V (ed) Associative Networks Academic Press, USA (1979) pp 51-92 12 Shnstri, L and Feidman, J A 'Semantic networks and neural nets' Technical Report TR131 Computer
Vol 1 No 5 December 1988
13 14
15 16 17
18 19 20 21 22 23 24 25 26
27 28 29 30 31 32
Science Department, University of Rochester, USA (June 1984) Winston, P H Artificial Intelligence (2nd) AddisonWesley, USA (1984) FIdalmu, S E 'Representing implicit knowledge' in Hinton, G E and Anderson, J A (ed~) Parallel Models of Associative Memory Erlbaum, USA (1981) pp 145-159 Brachmun, R J 'What IS-A is and isn't: an analysis of taxonomic links in semantic networks' Computer Vol 16 No 9 (October 1983) pp 30-36 Fahlman, S E 'Design sketch for a million-element NETL machine' AAAI Proc. of the Annual Conf. on Artif. Intell. (August 1980) pp 249-252 Fahlmun, S E, Hinton, G E and Sejnowski, T J 'Massively parallel architectures for AI: NETL, Thistle and Boltzmann machines' AAAI Proc. of the Annual Conf. on Artif. Intell. (August 1983) pp 109-113 HiUis, W D 'The Connection Machine: a computer architecture based on cellular automata' Physica Vol 10D (1984) pp 213-228 Waltz, D L 'Applications of the Connection Machine' Computer Vol 20 No 1 (January 1987) pp 85-97 Hilfis, W D and Steele, G L (Jr) 'Data parallel algorithms' Commun. ACM Vol 29 No 12 (December 1986) pp 1170-1183 Stanfill, C and Kahle, B 'Parallel free-text search on the Connection Machine' Commun. ACM Vol 29 No 12 (December 1986) pp 1229-1239 Frenkel, K A 'Evaluating two massively parallel machines' Commun. ACM Vol 29 No 8 (August 1986) pp 752-758 Uhr, L Multi-Computer Architectures for Artificial Intelligence John Wiley & Sons, USA (1987) Moldovan, D I and Tung, Y-W 'SNAP: a VLSI architecture for artificial intelligence' J. Parallel and Distrib. Comput. Vol 2 No 2 (1985) pp 109-131 Moldovan, D I and Wu, C I 'Parallel processing of a knowledge-based vision system' Proc. Joint Comput. Conf. (November 2-3 1986) pp 269-276 Dixit, V and Moldovan, D I 'Semantic Network Array Processor and its applications to image understanding' IEEE Trans. on Pattern Analysis & Machine Intell. Vol PAMI-9 No 1 (January 1987) pp 153-160 Sapaty, P S and Kocis, I 'A parallel network wave machine' (1986) Sapaty, P S 'A wave language for parallel processing of semantic networks' Comput. Artif. Intell. Vol 5 No 4 (1986) pp 289-314 Computer Special Issue on Dataflow Systems IEEE Computer Vol 15 No 2 (February 1982) Bic, L 'Processing of semantic nets on dataflow architectures' Artif. Intell. Vol 27 (1985) pp 219-227 Bic, L and Lee, C 'A data-driven model for a subset of logic programming' TOPLAS, ACM (1987) Wah, B W and Li, G-J 'A survey on special purpose computer architectures for AI' SIGART News. Vol 4 No 96 (April 1986) pp 28--46
265
Artificial intelligence (AI) applications are growing in several fields, and in many such applications knowledge bases must be manipulated. This activity is usually performed by an external agent such as a central processor, but often this cannot supply the speed required. Knowledge-oriented architectures provide an efficient execution of knowledge manipulations. This paper provides an introduction to a particular subset of knowledgeoriented architectures, the semantic network approach, which is one of the most commonly used methods of representing and manipulating knowledge in the A I field. A brief overview of the semantic network components is presented in order to provide a background to the topic. The purpose of this paper is to review the proposed, implemented and/or simulated architectures for semantic network processing, and to discuss the capabilities and limitations of such architectures. Keywords: semantic networks, parallel computers, artificial intelligence, knowledge-oriented systems
Knowledge representation and manipulation is required in many AI algorithms. Processing knowledge representations requires a large computational power that may not be delivered by conventional uni-processor architectures at a reasonable speed. Special purpose architectures are needed to manipulate such knowledge representations. The main goal of knowledge-oriented systems is to provide efficient execution of models for the representation and manipulation of knowledge 1. Knowledgeoriented architectures are divided into two approaches: symbolic manipulation and connectionism 2. Rule-based machines, such as DADO 3 and FAIM-14 are examples of symbolic manipulation architectures. In this approach, knowledge is represented as a collection of rules which are manipulated in order to make inferences. The connectionism approach is a class of massively parallel architectures where knowledge is distributed over the whole system s. A massively parallel architecture VLSI ResearchGroup, Department of Engineering Science,University of Oxford, Parks Road, OxfordOXI 3PJ, UK
manipulates knowledge by trying out different connections in the network until a set of solutions is found. The Connection Machine (CM) 6 and NETL 7 are examples of this approach. This paper deals with knowledgeoriented massively parallel connectionist systems, and in particular with semantic networks.
SEMANTIC NETWORKS Quillian s introduced the idea of a semantic network, although several proposals and changes have been made since (Brachman 9 provides an overview of these proposals). The basic principle of a semantic network is simple: each node of the interpreted graph represents a concept, and the links between them represent relationships between concepts ~°. Figure 1 shows such a semantic network, and the more important issues of semantic networks are briefly described below.
Hierarchical representation Knowledge in a semantic network is represented in a hierarchical form. The knowledge of whether or not an item belongs to a set is important for answering questions and retrieving facts 11. A method of providing a hierarchical representation is to use the arcs (or links) to classify the concepts represented by various nodes 1H3. An example of this method is shown in Figure 1; the
is-a
is-a
~
Taste
is-a
I
© (o.-,,)/
I
")
(Co~ou~.--~--~
S-a
~s-a
Figure I. A semantic network
0950-7051/88/050259-07 $03.00 © 1988 Butterworth & Co (Publishers)Ltd Vol I N o 5 December 1988
259
taxonomy is provided by means of arcs labelled 'IS-A'. For instance, in Figure 1 the arc between ELEPHANT and M A M M A L represents the knowledge that ELEPHANT is a subset of MAMMAL.
Inheritance As soon as humans beings identify something, many assumptions are made; for example, that birds fly, whales live in water, etc. A similar process can be done in semantic networks by means of inheritance. In a hierarchical representation, nodes at the lower levels inherit all the properties of nodes above them. The hierarchy determines the way that inheritance is passed; properties of a node do not affect the properties of nodes at higher levels. In Figure 1, for example, Clyde is described as an elephant; however, there is no direct link that determines Clyde's colour. By using inheritance, the colour of Clyde can be obtained from the elephant description. Inheritance from another semantic network tree must be disallowed in order to avoid contradictions; for instance, no node below PLANT can inherit anything that is below ANIMAL.
Implicit and explicit knowledge A human mind can store a great quantity and variety of knowledge, and can access whatever knowledge it needs very quickly and flexibly 14. For example, if it is stated that Clyde is an elephant, a human being can tell a great deal more about Clyde than what was stated explicitly. It is possible to infer (with a high degree of certainty) the colour, the number of legs and the size of Clyde. It is impossible to believe that all the knowledge about Clyde is stored in explicit form. The most feasible alternative is that knowledge is stored in an implicit form. A semantic network must be able to handle implicit knowledge since it would be almost impossible to store all facts explicitly. Figure 1, for example, shows how implicit knowledge is set in a semantic network. It can be seen that Clyde is an elephant, a mammal, an animal and a physical object by using implicit knowledge. Also, it can be observed that Clyde is a male and a veteran by using explicit knowledge representation. IS-A link Much representation of the world is concerned with the conceptual relations expressed in English sentences such as Peter is a bachelor and A dog is a domesticated mammal. The easiest way to get those statements into a semantic network scheme is to have a link that represents the IS-A part of such sentences I ~ IS-A connections form a hierarchy which makes semantic networks an efficient storage scheme. The IS-A link also allows the notion of inheritance of properties. There are a number of uses for an IS-A link 15, among them: • Subset/superset: one node is a superset of another node, e.g. the elephant set is a subset of mammals. • Generalization/specialization: some properties of nodes may be true for a node below it. This is usually conditional to definitions on the higher level node, providing there is no contradictory information. • Set membership: nodes that represent individuals are 260
set members of a higher level node, e.g. Clyde is a member of the ELEPHANT set. • Conceptual containment: a description includes another, e.g to be a triangle is to be a polygon with three sides; here a predicate is used in defining another. SEMANTIC NETWORK ARCHITECTURES Knowledge representations have been implemented on single processor architectures. This approach, however, presents a serious drawback: processing time is enormous due to the large knowledge bases and no a priori searching path. A simple query may lead to a search into many nodes in the knowledge tree. Some search-guiding heuristic algorithms may help to reduce the amount of searching. Such algorithms, however, are problem-domain dependent and carefully handcrafted to handle each particular set of problems 1.. On a serial machine this process takes a time proportional to the number of the nodes that have to be processed. With the advent of Very Large Scale Integration (VLSI), the possibilities of Ultra Large Scale Integration (ULSI), and the possible advantages of Wafer Scale Integration (WSI), a cost effective parallel architecture which can handle semantic networks may be achievable. Advantages of such an architecture include: • Time is proportional to the number of nodes in the branch that provides the answer. • Several alternatives are searched in parallel: if a query has more than one correct answer, they can be obtained with minimum extra effort. • Problem-domain independence: node and link processing is executed in parallel so there is no need for problem-domain heuristic algorithms. • Each node and link executes a small number of simple instructions. Several systems for semantic networks are described here, most of which have been implemented in hardware or simulated, though some are only proposals. NETL The NETL system was proposed by Fahlman 7''0'17. The NETL architecture consists of very simple hardware elements called 'node units'; relationships among concepts are represented by hardware elements called 'link units'. A serial controller is used to send simple instructions to node and link units by means of a shared bus. Figure 2 shows the basic NETL hardware. Each node has its own serial number, a few bits of information about the concept type that it represents, and a few bits to mark the nodes. Links also have a few bits for information about relationship type. Two wires come out of the link unit, wire A being connected to the object being described, and wire B going to the node that represents the class where the object has been classified. Nodes and links are able to perform simple Boolean operations. In a more recent version of NETL, a circuit switching network called 'hash net' is used to establish communication between a node and a link 16. Hashnet is a permutation network that is able to estabKnowledge-Based Systems
]
IIIIIIIIII
0Elephant, node unit
.Q
~.~
[
Hasnet
qs-a' link unit
I I I! I I I I II I
I
'Clyde' node unit
j
Serial control computer
Figure 2. NETL system hardware Thing
1 i
Router
1
Memory
Memory
Figure 4. VLSI module connection machine hardware Clyde
Figure 3. Marker propagation scheme for NETL (all solid arcs are 'is-a" links) lish one-to-one connections which are able to operate simultaneously. NETL can perform searches and inferences by means of a marker propagation scheme. In order to understand how the marker propagation scheme works, consider the example shown in Figure 3. Suppose that the colour of Clyde is needed. To start with, the controller sets the CLYDE node to marker one. Then all nodes above in the hierarchy (that are connected with IS-A links) are told to set a copy of marker one. In the next step the controller sends a command to all COLOUR-OF links with marker one and their A wire, telling them to set marker two in the node tied to their B wire. Finally, all nodes with marker two place their contents on the bus. The answer in the example is GREY. Fahlman found that the marker passing scheme may not be able to handle some recognition problems (such as vision, speech understanding, medical diagnosis, etc.) s. Value passing may be a better solution for this type of problem. Vol 1 No 5 December 1988
Connection Machine The Connection Machine (CM) system was proposed by Hillis6'1a (CM) and the prototype has been built by Thinking Machines Corporation (TMC). The CM can perform most operations at a rate in excess of 1000 MIPS 19. The TMC version of the full CM is intended to be a 1024 x 1024 PE array that is linked into a 14-cube. The system that has been built has 64K (256 x 256) synchronized 1-bit processors, each with a 4 Kbit memory. Figure 4 shows the architecture of the CM. The network of the CM was the prime effort of the machine design 6. The CM prototype has a 12-cube network with a packet switching scheme. Each VLSI chip has 16 processing elements and a router processor which is in charge of communicating between that PE cluster and the network. The CM is programmed in extensions of Common Lisp and C. The original purpose of the CM design was to manipulate knowledge stored in semantic networks concurrently xs. Each node and link was mapped onto a processing element and data was transferred by means of a message passing scheme. The resulting architecture, 261
Host computer
I Control logic
Memory Global bus
t
m
Cell
Cell
]
Instruction reg.
I
Mask reg.
I
h
]_]_" I
al bus
,I
Figure 5. Semantic network array processor (SNAP) architecture however, turned out to be a quite general purpose machine. This architecture executes data parallel algorithmsZ°: document retrieval from a large database 21, memory-based reasoning and natural language processing 19. In document retrieval, for example, from a database of 160 Mbyte the CM can operate at an effective execution speed of 6 000 MIPS zz. Another version of the CM is being developed at the MIT Artificial Intelligence Laboratory23. In this version the reconfiguring network is an NlogN network rather than a N-cube. This network needs logN stages to send data from one node to another.
Semantic Network Array Processor The Semantic Network Array Processor (SNAP) architecture is currently under study at the University of Southern California24. The architecture consists of a square array of identical processing dements which are connected in a Nearest Neighbour Interconnection (NNI) scheme for local communication, and in a column scheme for global communication (see Figure 5). The SNAP architecture is operated by a single controller which also acts as an interface between the array and a host computer. Each cell can be microprogrammed if it is required; this feature allows cells to operate independently. Each cell (or processing element) as shown in Figure 6, has three units: a Content-Addressable Memory (CAM), a Processing Unit (PU) and a Communication Unit (CU). The CAM has two modules: cell memory, which is the node local memory, and pointer memory, which is further subdivided to hold relation names and the addresses to which they relate. The PU controls both processing in the CAM and the operation of the CU. The PU has a reduced set of primitive (micro) instructions: AND, OR, NOT, SET, TEST, RESET, 262
MASK, MATCH, CAR and CDR. Data transfer between cells is executed by the CU. SNAP is capable of executing AI inference algorithms (i.e. deducing new facts from existing ones), since these algorithms are based on extensive searches and on a set of inference rules. The regularity of the SNAP array favours VLSI implementation. The performance of SNAP architecture has been observed by means of a simulation program which was written in UCI Lisp. SNAP simulation has revealed that little is gained in speed when eight nearest neighbour connections are allowed rather than four. This result is probably due to a low branching factor - - many semantic networks have a small number of links at each node. Also, it was observed that a slight improvement in processing time is achieved by a spiral allocation scheme. The SNAP system has been used to manipulate semantic networks in several applications, one of them being in high level computer vision25'26. Edge interpretation, and scene labelling are the main tasks in this application.
Parallel Network Wave Machine The Parallel Network Wave Machine (PNWM) system has been proposed by Sapaty and Kocis at the Slovak Academy of Sciences2L The main objective of the PNWM system is to obtain high performance on a parallel semantic network architecture by means of a distributed controller. Part of the control is sent to each node along with the data. A language called Wave was developed to achieve this control scheme 2s. The principle of this scheme is to propagate the formulae of this language in a dataflow mode, i.e. the program is sent along with data. The proposed PNWM architecture, shown in Figure 7, has five major blocks: a Language Processor (LP), a Planning and Control Processor (PCP), a Specialized Processor (SP), a Parallel Network Environment (PNE), and a Mass Memory (MM). The LP transforms user requests into internal wave language form. The PCP makes the initial activation of the PNE which is the main component of the system. The MM works as an extension of PE's local memory. The semantic networks are processed by the PNE block. The proposed PNE organization, shown in Figure 8, consists of a torus-like two-dimensional array with a four neighbour interconnection scheme. Each PE is also connected to two common buses which allow direct communication of any two processors in the same row or column. The PNWM is planned to be engaged in information systems, expert systems, CAD and modelling.
Dataflow approach The use of dataflow architectures29 for semantic network processing has been proposed by Bic30. In this approach, the semantic network is considered to be an interpreted dataflow graph. Each node is able to accept, process, and emit value tokens which are transferred asynchronously in the network. The semantic network is a dataflow graph as shown in Figure 9(a), where tl and t2 are constants in the nodes, and p is the label of the link (or arc) which is a constant. A query must also be a graph where at least one of the nodes or links corresponds to a constant while the others are variables. Figure 9(b) shows a query graph: T1, T2 and p can be either variables or constants. Knowledge-Based Systems
(i-1 ,j)
1
Cell (i,j)
IIO b u f f e r Global data bus
Global instruction bus
tl
CU
CAM
Busy full wait
Cell memory
Pocket r e g i s t e r
Prev pointer memory
t
I I n p u t queue
(i,j-1)
:
'Next' pointer memory
!6 general purpose flags
¢. re. :3 J2
,11,
O
t. r
iPV
:3 JD
-I~ ( i . j + l )
O
General purpose r e g i s t e r s Routing controller
Instruction memory
Control logic
I/O buffer
(i+I ,j)
Figure 6. SNAP processing element ~r user
r LP
PCP
SP
PNE
I
T
MM
b
Figure 7. Parallel network wave machine ( P N W M ) architecture The dataflow graph, representing either a semantic network or a query, must not have cycles, since those are difficult to detect due to the asynchronous scheme of dataflow systems. In order to have non-cyclic forms, the template must be transformed into a tree structure or a linear chain. Processing a linear template, such as the one shown in Figure 10, is performed thus: the template is sent as a token to all nodes in the semantic net that match Vol 1 N o 5 December 1988
Figure 8. P N W M system PNE module tI
t2
TI
T2
O
O
O
O
a
b
Figure 9. Dataflow ( a) semantic network, ( b) query graph 263
T°
Pi Ti+1 Pi+1
I
0
0
Ti+l
0
Pi+1
Pn-1 ---
T
n
0
Tn
.._ Pn-10
Figure 10. Linear graph template the leftmost node (T, in the template); nodes which receive this token will replicate it along all edges that match the first template edge Pl; these nodes then compare the second template/'2, and the operation continues until the last template is compared; all the values of the variables can then be obtained. All possible matches are found without the need for centralized control (a data-driven model of semantic network computations is presented in Reference 31).
SUMMARY There are two major approaches to parallel semantic network architectures: synchronous and asynchronous. The synchronous scheme is essentially a Single Instruction Multiple Data (SIMD) machine style. A central controller sends instructions to all PEs; some PEs may or may not execute each instruction, depending on their internal status. NETL, the CM, and SNAP are examples of this approach. Advantages of this approach include: simple processing elements, since almost no controller hardware is implemented in the PE; a simple communication scheme, because the controller only allows a few paths at any given time, thus avoiding collisions and higher PE density; and, since cells are small and regular, more of them may be put into a VLSI/WSI package. However, there may be some problems in a SIMD machine, such as some PEs possibly being idle for long periods of time when they are not in the tree that is being searched, and the central controller may be the bottleneck of the system, as nothing can be done until the controller indicates so. The asynchronous approach results in a Multi-Instruction Multi-Data (MIMD) streams machine style. A distributed controller, which is based on the dataflow philosophy, moves data and instructions around the PEs in the machine. The PNWM and Dataflow are examples of this approach. The MIMD architectures for semantic networks may provide the potential for high parallelism, since self-contained tokens may be processed simultaneously by several PEs. Problems in these architectures include the message passing overhead perhaps being quite large, complex processing elements are needed to process data and control flow, and the software to run these machines is complex. The 2-D array organization seems to be the preferred architecture for semantic network systems. Such an array has been used in the CM, SNAP and PNWM systems. The 2-D array advantages are a regular network, a well studied architecture, and a simple but powerful communication scheme for semantic network applications. Such advantages favour VLSI and WSI implemen264
Table 1. Semantic network processing systems
System
Developed at
Features
NETL
MIT
Simple PEs, single controller, node and link execute instructions
CM
Thinking Machines
64K PEs, 12-cube network, single controller
SNAP
USC
2-D array with global comms, scheme, nodes may be microprogrammed
PNWM
Slovak
2-D array with distributed system control
Dataflow UC, Irvine
Asynchronous scheme based on dataflow graphs
tations. Semantic networks with high node fanout may not be efficiently processed in a 2-D array; other types of networks may be useful, for example the n-cube and the cosmic cube. Simple processing elements are usually preferred over more complex ones, for instance, the PEs in NETL and CM architectures. This preference may be due to the simplicity of the semantic network instructions, and also to the need for having high PE density in the system. However, there are other approaches where the PEs are more powerful and execute complex instructions, e.g. the PEs in systems such as the SNAP, Dataflow and PNWM. The most common applications where semantic network engines have been used are knowledge retrieval and inferencing. Table 1 provides a summary of the semantic network architectures that have been described in this paper. CONCLUSION Artificial intelligence processing features 24'32 that influence the design of semantic network systems are: • Large problem size: the amount of information and the degree of parallelism are very large, therefore a great deal of computational power is required. • Memory access is intense: to speed up computation and avoid bottlenecks it is necessary to process data inside the memory. • Pattern retrieval operations are frequently used: parallel search may be the most promising approach to this operation. • Non-deterministic algorithms: there is no unique path from initial state to the goal. Parallel implementation of semantic nets is required in order to reduce processing time. The connectionism approach may be the most viable alternative to parallel semantic network processing. Several attempts towards a parallel semantic network have been presented in this paper. Most of them are based on simple processing elements connected, basically, in a two-dimensional array. A very large number of PEs are required in order to implement a semantic Knowledge-Based Systems
network system. A machine with so many active components cannot reasonably be expected to operate reliably without some form of fault-tolerances . However, few of these approaches have considered fault-tolerant schemes. Wafer-Scale Integration (WSI) technology provides some advantages (such as fast intermodule communication, higher reliability and lower pin-out count) which may benefit semantic network processing systems. WSI requires fault-tolerant schemes in order to overcome defects on the wafer, but a large system such as a knowledge-oriented architecture needs fault-tolerance anyway. Having a semantic network machine implemented in WSI may result in a faster, more reliable and possible cheaper system than having such a system in discrete VLSI chips. A WSI semantic network architecture with custom fault-tolerant schemes may have a great impact on the artificial intelligence field, in particular in areas where intense knowledge processing is needed. REFERENCES 1 Hwang, K, Gosh, J and Chowkwunyun, R 'Computer architectures for artificial intelligence' Computer Vol 20 No 1 (January 1987) pp 19-27 2 Treleaven, P C, Refenes, A N, Lees, K J and MeCabe, S C 'Computer architectures for artificial intelligence' Technical Report 119 (UCL-CS-TR 119) Department of Computer Science, University College London, UK (March 1986) 3 Stolfo, S J 'Initial performance of the DADO2 prototype' Computer Vol 20 No 1 (January 1987) pp 75-83 4 Anderson, J M, Coates, W S, Davis, A L, Hon, R W, Robinson, I N, Robinson, S V and Stevens, K S 'The architecture of FAIM-I' Computer Vol 20 No 1 (January 1987) pp 55-65 5 Fahlman,S E and Hinton, G E 'Connectionist architectures for artificial intelligence' Computer Vol 20 No 1 (January 1987)pp 100-109 6 Hillis, W D The Connection Machine MIT Press, USA (1985) 7 Fahiman, S E NETL: A System for Representing and Using Real-Worm Knowledge MIT Press, USA (1979) 8 Quillian, R M 'Semantic memory' in Minsky, M (ed) Semantic Information Processing MIT Press, USA (1968) 9 Brachmun, R J 'On the epistemological status of semantic networks' in Findler, N V (ed) Associative Networks Academic Press, USA (1979) pp 3-50 10 Woods, W A 'What's in a link: foundations for semantic networks' in Braehman, R J and Levesqne, H J (eds) Knowledge Representation MorganKaufmann, USA (1985) 11 Hendrix, G G 'Encoding knowledge in partitioned networks' in Findler, N V (ed) Associative Networks Academic Press, USA (1979) pp 51-92 12 Shnstri, L and Feidman, J A 'Semantic networks and neural nets' Technical Report TR131 Computer
Vol 1 No 5 December 1988
13 14
15 16 17
18 19 20 21 22 23 24 25 26
27 28 29 30 31 32
Science Department, University of Rochester, USA (June 1984) Winston, P H Artificial Intelligence (2nd) AddisonWesley, USA (1984) FIdalmu, S E 'Representing implicit knowledge' in Hinton, G E and Anderson, J A (ed~) Parallel Models of Associative Memory Erlbaum, USA (1981) pp 145-159 Brachmun, R J 'What IS-A is and isn't: an analysis of taxonomic links in semantic networks' Computer Vol 16 No 9 (October 1983) pp 30-36 Fahlman, S E 'Design sketch for a million-element NETL machine' AAAI Proc. of the Annual Conf. on Artif. Intell. (August 1980) pp 249-252 Fahlmun, S E, Hinton, G E and Sejnowski, T J 'Massively parallel architectures for AI: NETL, Thistle and Boltzmann machines' AAAI Proc. of the Annual Conf. on Artif. Intell. (August 1983) pp 109-113 HiUis, W D 'The Connection Machine: a computer architecture based on cellular automata' Physica Vol 10D (1984) pp 213-228 Waltz, D L 'Applications of the Connection Machine' Computer Vol 20 No 1 (January 1987) pp 85-97 Hilfis, W D and Steele, G L (Jr) 'Data parallel algorithms' Commun. ACM Vol 29 No 12 (December 1986) pp 1170-1183 Stanfill, C and Kahle, B 'Parallel free-text search on the Connection Machine' Commun. ACM Vol 29 No 12 (December 1986) pp 1229-1239 Frenkel, K A 'Evaluating two massively parallel machines' Commun. ACM Vol 29 No 8 (August 1986) pp 752-758 Uhr, L Multi-Computer Architectures for Artificial Intelligence John Wiley & Sons, USA (1987) Moldovan, D I and Tung, Y-W 'SNAP: a VLSI architecture for artificial intelligence' J. Parallel and Distrib. Comput. Vol 2 No 2 (1985) pp 109-131 Moldovan, D I and Wu, C I 'Parallel processing of a knowledge-based vision system' Proc. Joint Comput. Conf. (November 2-3 1986) pp 269-276 Dixit, V and Moldovan, D I 'Semantic Network Array Processor and its applications to image understanding' IEEE Trans. on Pattern Analysis & Machine Intell. Vol PAMI-9 No 1 (January 1987) pp 153-160 Sapaty, P S and Kocis, I 'A parallel network wave machine' (1986) Sapaty, P S 'A wave language for parallel processing of semantic networks' Comput. Artif. Intell. Vol 5 No 4 (1986) pp 289-314 Computer Special Issue on Dataflow Systems IEEE Computer Vol 15 No 2 (February 1982) Bic, L 'Processing of semantic nets on dataflow architectures' Artif. Intell. Vol 27 (1985) pp 219-227 Bic, L and Lee, C 'A data-driven model for a subset of logic programming' TOPLAS, ACM (1987) Wah, B W and Li, G-J 'A survey on special purpose computer architectures for AI' SIGART News. Vol 4 No 96 (April 1986) pp 28--46
265