- Email: [email protected]
Overcoming Communication Bottlenecks in Distributed Intelligent Control of FMS
MANUFACTURING NE1WORKS
Copyright © IFAC Information Control Problems in Manufacturing Technology, Toronto, Canada, 1992
OVERCOMING COMMUNICATION BOTTLENECKS IN DISTRIBUTED INTELLIGENT CONTROL OF FMS D.S. Devapriya, B. Descotes-Genon and P. Ladet Laboraloire d'AuJomalique de Grenoble. EN.SIE.G., B.P. 46,38402 SI . Marlin d'Heres, France
Abstract. An approach to reduce the effect of problems related to inter-node communication in Distributed Problem Solvers (DPS) is presented for the special case of real-time control of Computer Integrated Manufacturing Systems (CIMS) . Some techniques based on the distribution of control and VLSI implementations of control Petri nets were presented in (DEVAPRIYA, 91c; 92a). The present paper discusses the necessity for furnishing the system with a panoplie of diverse communication techniques and letting the system learn to use the best technique for a given situation. This permits a high degree of flexibility of the system without sacrificing unduely on communication efficiency while avoiding bottleneck situations in the DPS network. Key Words. Distributed Problem Solving, Distributed Artificial Intelligence, Open Systems, Blackboard, Actors, Petri Nets, Object-Oriented Programming, Manufacturing Systems, SCOPE. 1. INTRODUCTION
The communication protocols could be considered at two levels: low-level network protocols are concerned with naming, protection, synchronization, atomic transactions etc., and the application level protocol is concerned with the definition of choices like, whether the messages are targetted to a specific receiver or they are broadcast. If the messages are targetted, the paradigm for communication can be either shared memory, message passing or some combination of the two.
A Distributed Intelligence architecture (SCOPE) was proposed in (DEVAPRIYA, 1990; 1991a; b) for the real-time control of Flexible Manufacturing Systems (FMS). FMS refers to flexible, programmable, integrated automation, a special case of CIMS. A major challenge for knowledge based systems being pattern matching over a large number of objects and the relationship between these objects, parallelism has been suggested as the most feasible alternative to achieve the computational requirements for real-time applications. In a distributed system such parallelism could be exploited at two levels; if such a system is considered as a network of semi-independent intelligent nodes, inter-node parallelism and intranode parallelism.
Classic examples of architectures using targetted messages are, the blackboard framework where they are targetted indirectly through a shared global memory, and the actors framework using messages targetted directly to receiver nodes . In the contract net framework (DAVIS, 1983) both these protocols (targetted and broadcast) are used. The problem is that once the system is designed, the selected communication techniques become constraints on system functioning; their selection is no more a decision to be made in the system. However there can arise situations where the optimal technique would be diffrent from that fixed in the architecture.
Given the determining role played by the nature of communication in the functioning of Distributed Intelligence systems, particular care has to be taken in selecting appropriate mechanisms. The key aspects to be considered in this respect have been identified as, the paradigm by which communication takes place, the semantic content of the information, and the protocols adopted to effectively use the limited bandwidth available.
The main idea pushed in the current paper is that it is essential to avoid too early commitments to particular techniques of communication. One finds too often that a given architecture is characterized by the communications techniques it has adopted. This should not be the case.
The content of communications has to be relevant, timely and complete (DURFEE, 1985). Relevance here is a measure of how consistent a message is with what is already known; timeliness, a measure of how much a message will affect a receipient; and completeness measures how much of a solution is being sent. The content of the communication can be uncertain and incomplete. It may not be completely accurate . Communication of partial results that are at least consistent with the functional semantics of an agent has been named functionally accurate communication (LESSER, 1981).
In the distributed processing field we can see considerable efforts being made for constructing open systems as a response to the opportunities being opened up by new advances in hardware. In (DEVAPRIYA, 91c) we discussed the implications of these developments to distributed problem solving in general and SCOPE in particular. We wish to continue
257
this discussion in the current paper. The organizational and technological aspects of the proposed open distributed problem solving architectures were introduced in (DEV APRIY A. 91c; 92a). In § 2 we will give a brief introduction to SCOPE. The communication techniques proposed for our architecture will be presented in § 3. In § 3.1 we give details of a new communication technique we have introduced result-sharing cooperation.
with a series of such techniques. These are presented next. 3. INTERNODE COMMUNICATION MECHANISMS IN SCOPE 3.1. Communication usin~ arcs subject to nonsimultaneity constraints For the process of matching local solutions of two subnodes by a root node we have introduced a special communication paradigm; dynamic creation of Petri net arcs bound by a non-simultaneity constraint (DEVAPRIYA. 91b). Two arcs bound by a nonsimultaneity constraint cannot exist together. All arcs not bound by such constraints can exist simultaneously. In this mode of communication. when two or more Petri nets are connected by a set of arcs. the composite Petri net is allowed to evolve autonomously until it reaches a stable marking before these temporary arcs are removed. leaving behind the evolved marking in the individual Petri nets. This communication is controlled by the specification OBCS of the parent node or that of the matching node if the latter node is not related to them by hierarchical links.
2. DISTRIBUTED INTELUGENT CONTROL OF FMS SCOPE is a network of cooperating intelligent nodes that constitute a distributed controller kernal for Flexible Manufacturing Systems. It is intended to be easily adapted for different manufacturing systems and we propose this as a basis for the standardization of control architectures from the point of view of control system designers. We have a network of problem solving nodes and two service nodes. themselves distributed. A problem solving node can be a planning node. an operational control node or a coordinator node. Generally a planning node deals with a single class of objects (product·type object class. for instance). An operational control node or a coordinator node is a macro-node consisting of a number of sub·nodes (sequencing node which has the two sub-nodes, product-node and machine-node. for instance). The first service node (demand-class node) does the event treatment and the second. the task control (productfamily planning task etc.). The presence of service nodes is explained by the fact that we use the opportunistic problem solving paradigm first introduced in blackboard type architectures (NIl. 86).
A situation which we see as a natural candidate for this communication mechanism is the case where one has to deal with configurable resources which could serve several consumers which could potentially choose among several resources. In a distributed application of this type. the configurable resources (each a potential possessor of a processor) could be expected to use local information to compute a locally optimized configuration. On the other hand each consumer would use its local information to make the best assignment decision for its service needs. Naturally. for such a resource or a consumer to have its own local processing ability, one would expect them to be sufficiently sophisticated (having sufficiently complex functions) agents. In this case a primary matching function demanded by any cooperative decision strategy could consist of verifying which of the elements of a given resource configuration are eventually used by some consumer, and which of the service needs remain unsatisfied given the locally decided resource configurations. This function would be called repeatedly at different stages of solution development until a satisfactory solution is found.
Each node consists of the object class (or classes) that it deals with. its local model (which describes the dynamics of the node) and a set of problem solving agents (knowledge sources) that could be invoked by this node. The object class(es) is represented by a Petri net with an object oriented interpretation (PNO) and acts as a structured local data base or a local hypothesis register. The local model indicates the conditions under which this problem solving node could invoke any of its agents. and also. when a node demands the services of another node. this demand is formulated through the local model. These two functions correspond to the specification and the implementation Object Control Structures (OBCS).
The basic structure needed for this mechanism is given in figures 3.1 and 3.2 using ordinary Petri nets. Figure 3.1 gives the generic Petri net structures for a resource agent and a consumer agent.
Each node can communicate with its immediate environment. Each node has substantial processing capability and the internode communication is limited to a minimum. as opposed to distributed processing. The choice of a comunication paradigm between two given nodes is dependent on the content of the messages to be sent. the expectation that a message (of a given class) originating at one is intended for the other. the expectation that one is waiting for messages (of a particular class) from the other, the same expectations with respect to a third node, and available (accessible) hardware. The variances of the expectations mentioned would decrease with more experiance in using the control system. Thus it is natural to expect that experience with the controller might change initial choices for communication mechanisms. In view of anticipating this adaptation of communication mechanisms we have furnished SCOPE
Con~umer
Resourc:e
PIOO
PI02
TIOO
TI02
PIOI
•
«onflgured elements of the resource>
PIOO
TIOO
Figure 3.1. Generic structures for a resource agent and a consumer agent using ordinary Petri nets.
258
. -- - --------- "\ Coa.umer
Resource
,
PI02
nos
,. -.........
As can be seen from fig. 3.2, it is necessary to add one place, two transitions and eight arcs for matching the local solutions of two agents . Hence, for several cooperating agents this does not appear to be very attractive. This position changes radically when we take the combination of coloured Petri nets and objectoriented interpretation to implement the same communication mechanism. The coloured Petri net implementation is as follows (fig. 3.3).
. .. .... f6
~ ~ ~ -" ~
Pl~
." -" ,.,
_______ _
I ••• ·)
.~#
,.# #
.
Ib.b·)
Pl03 Id)
n
no Id)
Figure 3.3. Coloured Petri net structure representing the matching of locally configured elements of a resource and local resource assignments for service needs of a consumer.
~
0
P201
-f2
-f3
(17-f6)
0
P202
0
0
- [5
f9
P203
0
0
f8
-fIO
All these communication methods presume that the agents taking part are already identified when the communication is initiated . This condition is verified if one of the agents has the intelligence for identifying its next communicating partner, or else if there is some communications manager who directs the interaction between them. Otherwise we have to develop a mechanism for this purpose.
f3 : a* ~ a*; f4: a ~ e; f4: a* ~ e ; f5 : b ~ b ; f5 : b* ~ b; f6: b ~ a ; f6 : b* ~ a* ; fl : b ~ a*; fl: b* ~ a* ; d; f9: d
0
One could directly use UNIX 'pipe' for communicating between processes. We use this for example for activating a Petri net transition based on an event in a different process (a UNIX process).
f1:a~e; f2:a~a; f3:a~a;
~
f4
Another mode of communication between two problem solving nodes is using the classical message passing facility in an object-oriented programming environment. Messages of the format could be sent from an object to another. Complex messages could be constructed using this facility, but if one is dealing with parallel objects. the synchronization has to be done by the programmer and this can demand considerable work.
Then the functions on the arcs have the following interpretations,
d ; f8 : b*
-fI
3.2. Message passing
The figures in parantheses represent the colour sets attached to nodes of the Petri net. The set {e} represents the unconfigured elements of the resource. {6} is a possible configuration for an element. Then, {6"'} denotes a con figured resource element already having satisfied a service need of a consumer (a configuration that has already proved to be compatible with some resource assignment to a service need - we call such a configuration a proved configuration -). {b} represents an unsatisfied service need, and {b"'} is a copy of {b} which could use a proved configuration. {d} represents a service need that has already been matched with a configured resource element.
~
P200
Now, in SCOPE we define each agent as an object (in the object-oriented programming sense); the generic Petri net structure representing an agent is an object which has attributes and methods and which can have multiple inheritance. In order to implement a communication between two objects one has to construct a new object using instances of the two existing objects and two arcs to be defined. Once the desired evolution of the new object has reached a stable state, the constituant objects could be re-established by removing the two arcs introduced earlier. In the object-oriented environment the programming effort required for this implementation is minimal.
Consumer Ib)
1'200
f8 : b
T203
The non-simultaneity constraints on the connecting arcs comes from the fact that one service need will not be assigned more than one resource in a given configuration, whereas a given resource could be assigned to several service needs in the same resource configuration, first because the service needs could be treated sequentially and second. there are potentially more than one configurable element in a resource.
f4
I ••• ·)
T202
The only new element introduced due to the establisment of the communication is the element (2,3). i.e. (fl -f6) . The other elements contain information on the two generic nodes. This representation clearly reflects the fact that these are two communicating agents.
Figure 3.2. Ordinary Petri net structure representing the matching of locally configured elements of a resource and local resource assignments for service needs of a consumer.
P202
T201
Figure 3.4. The incidence matrix of the Petri net of fig. 4.6.
' -_ _ _ _ _ _ _-' (conOgured re.oune - TlOJ elemen •• Ire.dy h."lng .erved _ service nee")
Ruource
T200
3.3. Environment of a node
b;
f10:d~d.
A possible mechanism is illustrated in fig. 3.5 (we first presented these structures in (DEVAPRIYA, 1991a; b» which corresponds to opportunistic problem solving
Hence the incidence matrix of the connected pair of Petri nets is,
259
open nature of our architecture we can benefit from these concepts to further improve communication efficiency.
implemented according to the blackboard paradigm. Here a node calling for the treatement of its local hypothesis modification does not know the node which will respond. Inside a node also its KSs (knowledge sources) are disconnected (uncoupled) and react in an opportunistic manner. This control structure permits parallel activation of KSs. The specification of KSs permits the avoidance of conflictual situations. We see the intervention of the two service nodes (demand-class and task-class) in this communication.
3.4. Distributed blackboards Distributed blackboard systems represent an important research effort to realize real-time response, limited communication bandwidth and reliability in distributed problem solving systems. This model is built as a network of nodes, each of which is a complete blackboard system (like Hearsay-H) (ERMAN, 80). Any given node has direct communication capability with its neighbouring nodes. Any hypothesis modification on the blackboard of a node will be assessed locally for its impact on the activity of the neighbouring nodes. If the impact measure is sufficiently high, then an abstraction of this hypothesis modification would be broadcast by the TRANSMIT knowledge source of the node to all the nodes of its neighbourhood. The RECEIVE knowledge source of a receiving node would integrate the received information onto its own blackboard.
Let us look at fig. 3.5 a little closer. This shows how a partial result posted at a node could invoke KSs of another node. This posting done in the object-part of the node. It could be the pre-condition in a rule of the Implementation Object Control Structure (Imp.OBCS) of this planning node, which has as action part, a service demand. Such a demand consists of a message to the corresponding demand-class sub-node (since the demand-class node is distributed) calling for a service, and hence it is received at an input port of the Spec.OBCS of the node. The role of the Spec.OBCS is to indicate, depending on the state of the objectls of its node, which of its KSs can respond to this message. The termination effect of the possible activation of a KS (according to the corresponding KS specification) is posted on the object/s of the demand-class node. This posting could result in the formulation of a service demand by the Imp.OBCS of the node for the creation of a task instance. Note that there has been no direct service given to the calling node, and hence this is one way communication. When this service demand, in turn, is received at the task-class sub-node (this node controls tasks) it could lead to the creation of the task instance at the object of this node . For the implementation of this task, the Imp.OBCS of the task-class node sends a service demand to the nodes (or node) that specialize(s) on this plannning problem (node 2 in this case). This could be done using message passing or broadcasting.
This mode of problem solving reduces the need for synchronization of information transmission among nodes and the processing order within nodes. This reduction in explicit synchronization has the extremely favourable effect of reducing the response time of the system, because it increases the possibility of parrallel processing. The communication bandwidth could be expected to be reduced due to the possibility to achieve effective internode cooperation using abstract information. In order to increase network reliability, a knowledge based mechanism called murmuring is used for a node to retransmit high-impact information if during a specified time interval it neither receives nor generates higher impact information. Murmuring can be used to correct for lost communications.
We saw that the communication between a service calling node and the demand-class node is one way; from the former to the latter. The communication between the demand-class node and the task-class node is equally one way ; from the former to the latter. On the other hand, the task-class node and the service giving node have a two way communication.
This mecanism is different from that given in fig. 3.5 in that the former uses broadcasting exclusively, whereas the latter is not costrained in this choice (it can use message passing non-simultaneity constraint arcs etc. in addition). SCOPE is intended to provide the facility of dynamic binding the communication mechanism among nodes . This is an important learning function neglected in many of the earlier distributed problem solving architectures.
Yet there are some important problems to be resolved. The data base accesible to service nodes in the structure has to be defined. Do they have access to the databases of all the nodes whose local solutions are matched by them? How does this help to overcome communication bottlenecks?
Given the choice of Petri nets interpreted as objects as the computational model of SCOPE, and the extensive use of pattern matching in this interpretation, we tried to benefit from advances made in the parallel implementation of distributed AI software «DELGAOOFRIAS, 90), (QIU, 90). We presented some results in this direction in (DEV APRIYA, 91c and 92a). Notably we gave a method for implementing high level Petri nets using associative memory.
In fact we have eliminated one souce of communication bottlenecks by distributing the data base and the control among the nodes. We use the structure of the problem to identify a neighbourhood for each node. This is done based on direct access to distributed specializations of the unique nodes. A node that can directly call for services from a demand-class sub-node of a given specialization would have for its neighbourhood, the nodes that can be directly invoked for executing the tasks generated by the task-class sub-node of the same specialization.
With an implementation of the more complex nodes using highly parallel CAM chips, SCOPE could benefit from the flexibility offered by opportunistic KS activation while effectively overcomming risks of communication bottlenecks. It is popssible to implement a class of problem solving nodes that contain KSs whose pre-conditions, trigger-conditions etc. could be periodically matched against the marking of the Petri net objects in the task-class node. This is
Work on distributed blackboards has resulted in the definition of some key notions in inter-node communication (NIl, 89). Without compromising the
260
Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1991b). Petri net based node structures for distributed problem solving in FMS control. to appear in The International Journal on Computer Integrated Manufacturing Systems. Butterworth Publ.. U.K .• 1991. Devapriya, D. S .• B. Descotes-Genon and P. Ladet (1991c). Issues in overcoming communication bottlenecks in an opportunistic distributed problem solving architecture. In the proceedings of The International Workshop on Open Distributed Processing. Berlin. Germany. October 8 -11. 1991. Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1992a). Petri nets interpreted as objects for distributed intelligent control: implementation issues. SICICI '92. Singapore International Conference on Intelligent Control and Instrumentation. Singapore. February 17 - 21 . 1992 . Durfee. E. H.. V. R. Lesser and D. D. Corkill. (1985). Coherent cooperation among communicating problem solvers . In Proc. 1985 Distributed Artificial Intelligence Workshop. pp. 231-276. Dec. 1985. Erman. L. D.• F. Hayes-Roth. V. R. Lesser and D. R. Reddy (1980) . The Hearsay-I! speechunderstanding system: integrating knowledge to resolve uncertainty. Computing Surveys. 12. 213 - 253. 1980. Hewitt. C. and H. Lieberman. (1984). Design issues in parallel architectures for artificial intelligence. Proc. 28th IEEE Computer Soc. Int. ConL. San Francisco. CA. Feb. 1984. pp. 418-423 . Lesser. V. R. and D. D. Corkill. (1981). Functionally accurate. cooperative distributed systems. IEEE Trans. on Systems. Man and Cybernetics, Vol. SMC-11. No.l. January 1981. pp. 81-96. Nii. H. P. (1986). Blackboard systems. blackboard application systems from a knowledge engineering perspective. The AI Magazine. August. 1986. Nii. H. P. (1989). Experiments on Cage and Poligon: measuring the performance of parallel blackboard systems. Distributed Artificial Intel!jgence. Vol.
possible since the KSs are specified using Petri nets and the nodes are implemented as coloured Petri nets and hence their state is represented by their marking. The marking vector of any of these Petri nets can be stored in a CAM word. The parallel matching of an incomming marking vector with the vectors stored in the chip can provide the necessary control for the realtime invocation of Knowledge Sources. With this implementation technique. the nonincrementality of the Petri net token player is no more a constraint for the use of high level Petri nets in realtime control. 4. CONCLUSION Dynamic binding of the communication paradigm between cooperating problem solving nodes in a Distributed Problem Solver is introduced as a measure for overcoming communication bottlenecks in such architectures. This approach is complementary to our earlier propositions on the distribution of control and VLSI based implementation of complex Petri net structures. A new technique is detailed for a type of result sharing cooperation. These results are expected to help in a better adaptation of high level Petri nets as a computational model for the real-time control of Integrated Manufacturing Systems. REFERENCES Davis. R. and R. G . Smith. (1983). Negotiation as a metaphor for distributed problem solving . Artificial Intell .. Vol. 20. pp. 63-109. January 1983. Delgado-Frias. J. G . and W. R. Moore. (1990). A semantic network architecture for knowledge base systems. Eng. Appli. of AI. Vol. 3. March. 1990. pp. 4-10. Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1990) . A Petri net based blackboard type architecture for FMS control (SCOPE). APMS'90. International Conference On Advances In Production Management Systems. IFIP W. G . 5.7. Dipoli. Espoo. Finland. August 1990. Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1991a). Distributed intelligence systems for FMS control using objects modelled with Petri nets (SCOPE Blackboard). IFAC Symp. on Distributed Intelligence Systems. Arlington. Virginia. U.S.A .• August. 1991.
n. Qiu. G. and J. D. Birdwell. (1990). Distributed realtime expert control system. In Proceedings of the Conference on Decision and Control. Honolulu. Hawaii. Dec. 1990. pp. 1525-1530.
NocIol
NocIo2
Figure 3.5. Communicating nodes. 261
Copyright © IFAC Information Control Problems in Manufacturing Technology, Toronto, Canada, 1992
OVERCOMING COMMUNICATION BOTTLENECKS IN DISTRIBUTED INTELLIGENT CONTROL OF FMS D.S. Devapriya, B. Descotes-Genon and P. Ladet Laboraloire d'AuJomalique de Grenoble. EN.SIE.G., B.P. 46,38402 SI . Marlin d'Heres, France
Abstract. An approach to reduce the effect of problems related to inter-node communication in Distributed Problem Solvers (DPS) is presented for the special case of real-time control of Computer Integrated Manufacturing Systems (CIMS) . Some techniques based on the distribution of control and VLSI implementations of control Petri nets were presented in (DEVAPRIYA, 91c; 92a). The present paper discusses the necessity for furnishing the system with a panoplie of diverse communication techniques and letting the system learn to use the best technique for a given situation. This permits a high degree of flexibility of the system without sacrificing unduely on communication efficiency while avoiding bottleneck situations in the DPS network. Key Words. Distributed Problem Solving, Distributed Artificial Intelligence, Open Systems, Blackboard, Actors, Petri Nets, Object-Oriented Programming, Manufacturing Systems, SCOPE. 1. INTRODUCTION
The communication protocols could be considered at two levels: low-level network protocols are concerned with naming, protection, synchronization, atomic transactions etc., and the application level protocol is concerned with the definition of choices like, whether the messages are targetted to a specific receiver or they are broadcast. If the messages are targetted, the paradigm for communication can be either shared memory, message passing or some combination of the two.
A Distributed Intelligence architecture (SCOPE) was proposed in (DEVAPRIYA, 1990; 1991a; b) for the real-time control of Flexible Manufacturing Systems (FMS). FMS refers to flexible, programmable, integrated automation, a special case of CIMS. A major challenge for knowledge based systems being pattern matching over a large number of objects and the relationship between these objects, parallelism has been suggested as the most feasible alternative to achieve the computational requirements for real-time applications. In a distributed system such parallelism could be exploited at two levels; if such a system is considered as a network of semi-independent intelligent nodes, inter-node parallelism and intranode parallelism.
Classic examples of architectures using targetted messages are, the blackboard framework where they are targetted indirectly through a shared global memory, and the actors framework using messages targetted directly to receiver nodes . In the contract net framework (DAVIS, 1983) both these protocols (targetted and broadcast) are used. The problem is that once the system is designed, the selected communication techniques become constraints on system functioning; their selection is no more a decision to be made in the system. However there can arise situations where the optimal technique would be diffrent from that fixed in the architecture.
Given the determining role played by the nature of communication in the functioning of Distributed Intelligence systems, particular care has to be taken in selecting appropriate mechanisms. The key aspects to be considered in this respect have been identified as, the paradigm by which communication takes place, the semantic content of the information, and the protocols adopted to effectively use the limited bandwidth available.
The main idea pushed in the current paper is that it is essential to avoid too early commitments to particular techniques of communication. One finds too often that a given architecture is characterized by the communications techniques it has adopted. This should not be the case.
The content of communications has to be relevant, timely and complete (DURFEE, 1985). Relevance here is a measure of how consistent a message is with what is already known; timeliness, a measure of how much a message will affect a receipient; and completeness measures how much of a solution is being sent. The content of the communication can be uncertain and incomplete. It may not be completely accurate . Communication of partial results that are at least consistent with the functional semantics of an agent has been named functionally accurate communication (LESSER, 1981).
In the distributed processing field we can see considerable efforts being made for constructing open systems as a response to the opportunities being opened up by new advances in hardware. In (DEVAPRIYA, 91c) we discussed the implications of these developments to distributed problem solving in general and SCOPE in particular. We wish to continue
257
this discussion in the current paper. The organizational and technological aspects of the proposed open distributed problem solving architectures were introduced in (DEV APRIY A. 91c; 92a). In § 2 we will give a brief introduction to SCOPE. The communication techniques proposed for our architecture will be presented in § 3. In § 3.1 we give details of a new communication technique we have introduced result-sharing cooperation.
with a series of such techniques. These are presented next. 3. INTERNODE COMMUNICATION MECHANISMS IN SCOPE 3.1. Communication usin~ arcs subject to nonsimultaneity constraints For the process of matching local solutions of two subnodes by a root node we have introduced a special communication paradigm; dynamic creation of Petri net arcs bound by a non-simultaneity constraint (DEVAPRIYA. 91b). Two arcs bound by a nonsimultaneity constraint cannot exist together. All arcs not bound by such constraints can exist simultaneously. In this mode of communication. when two or more Petri nets are connected by a set of arcs. the composite Petri net is allowed to evolve autonomously until it reaches a stable marking before these temporary arcs are removed. leaving behind the evolved marking in the individual Petri nets. This communication is controlled by the specification OBCS of the parent node or that of the matching node if the latter node is not related to them by hierarchical links.
2. DISTRIBUTED INTELUGENT CONTROL OF FMS SCOPE is a network of cooperating intelligent nodes that constitute a distributed controller kernal for Flexible Manufacturing Systems. It is intended to be easily adapted for different manufacturing systems and we propose this as a basis for the standardization of control architectures from the point of view of control system designers. We have a network of problem solving nodes and two service nodes. themselves distributed. A problem solving node can be a planning node. an operational control node or a coordinator node. Generally a planning node deals with a single class of objects (product·type object class. for instance). An operational control node or a coordinator node is a macro-node consisting of a number of sub·nodes (sequencing node which has the two sub-nodes, product-node and machine-node. for instance). The first service node (demand-class node) does the event treatment and the second. the task control (productfamily planning task etc.). The presence of service nodes is explained by the fact that we use the opportunistic problem solving paradigm first introduced in blackboard type architectures (NIl. 86).
A situation which we see as a natural candidate for this communication mechanism is the case where one has to deal with configurable resources which could serve several consumers which could potentially choose among several resources. In a distributed application of this type. the configurable resources (each a potential possessor of a processor) could be expected to use local information to compute a locally optimized configuration. On the other hand each consumer would use its local information to make the best assignment decision for its service needs. Naturally. for such a resource or a consumer to have its own local processing ability, one would expect them to be sufficiently sophisticated (having sufficiently complex functions) agents. In this case a primary matching function demanded by any cooperative decision strategy could consist of verifying which of the elements of a given resource configuration are eventually used by some consumer, and which of the service needs remain unsatisfied given the locally decided resource configurations. This function would be called repeatedly at different stages of solution development until a satisfactory solution is found.
Each node consists of the object class (or classes) that it deals with. its local model (which describes the dynamics of the node) and a set of problem solving agents (knowledge sources) that could be invoked by this node. The object class(es) is represented by a Petri net with an object oriented interpretation (PNO) and acts as a structured local data base or a local hypothesis register. The local model indicates the conditions under which this problem solving node could invoke any of its agents. and also. when a node demands the services of another node. this demand is formulated through the local model. These two functions correspond to the specification and the implementation Object Control Structures (OBCS).
The basic structure needed for this mechanism is given in figures 3.1 and 3.2 using ordinary Petri nets. Figure 3.1 gives the generic Petri net structures for a resource agent and a consumer agent.
Each node can communicate with its immediate environment. Each node has substantial processing capability and the internode communication is limited to a minimum. as opposed to distributed processing. The choice of a comunication paradigm between two given nodes is dependent on the content of the messages to be sent. the expectation that a message (of a given class) originating at one is intended for the other. the expectation that one is waiting for messages (of a particular class) from the other, the same expectations with respect to a third node, and available (accessible) hardware. The variances of the expectations mentioned would decrease with more experiance in using the control system. Thus it is natural to expect that experience with the controller might change initial choices for communication mechanisms. In view of anticipating this adaptation of communication mechanisms we have furnished SCOPE
Con~umer
Resourc:e
PIOO
PI02
TIOO
TI02
PIOI
•
«onflgured elements of the resource>
PIOO
TIOO
Figure 3.1. Generic structures for a resource agent and a consumer agent using ordinary Petri nets.
258
. -- - --------- "\ Coa.umer
Resource
,
PI02
nos
,. -.........
As can be seen from fig. 3.2, it is necessary to add one place, two transitions and eight arcs for matching the local solutions of two agents . Hence, for several cooperating agents this does not appear to be very attractive. This position changes radically when we take the combination of coloured Petri nets and objectoriented interpretation to implement the same communication mechanism. The coloured Petri net implementation is as follows (fig. 3.3).
. .. .... f6
~ ~ ~ -" ~
Pl~
." -" ,.,
_______ _
I ••• ·)
.~#
,.# #
.
Ib.b·)
Pl03 Id)
n
no Id)
Figure 3.3. Coloured Petri net structure representing the matching of locally configured elements of a resource and local resource assignments for service needs of a consumer.
~
0
P201
-f2
-f3
(17-f6)
0
P202
0
0
- [5
f9
P203
0
0
f8
-fIO
All these communication methods presume that the agents taking part are already identified when the communication is initiated . This condition is verified if one of the agents has the intelligence for identifying its next communicating partner, or else if there is some communications manager who directs the interaction between them. Otherwise we have to develop a mechanism for this purpose.
f3 : a* ~ a*; f4: a ~ e; f4: a* ~ e ; f5 : b ~ b ; f5 : b* ~ b; f6: b ~ a ; f6 : b* ~ a* ; fl : b ~ a*; fl: b* ~ a* ; d; f9: d
0
One could directly use UNIX 'pipe' for communicating between processes. We use this for example for activating a Petri net transition based on an event in a different process (a UNIX process).
f1:a~e; f2:a~a; f3:a~a;
~
f4
Another mode of communication between two problem solving nodes is using the classical message passing facility in an object-oriented programming environment. Messages of the format
Then the functions on the arcs have the following interpretations,
d ; f8 : b*
-fI
3.2. Message passing
The figures in parantheses represent the colour sets attached to nodes of the Petri net. The set {e} represents the unconfigured elements of the resource. {6} is a possible configuration for an element. Then, {6"'} denotes a con figured resource element already having satisfied a service need of a consumer (a configuration that has already proved to be compatible with some resource assignment to a service need - we call such a configuration a proved configuration -). {b} represents an unsatisfied service need, and {b"'} is a copy of {b} which could use a proved configuration. {d} represents a service need that has already been matched with a configured resource element.
~
P200
Now, in SCOPE we define each agent as an object (in the object-oriented programming sense); the generic Petri net structure representing an agent is an object which has attributes and methods and which can have multiple inheritance. In order to implement a communication between two objects one has to construct a new object using instances of the two existing objects and two arcs to be defined. Once the desired evolution of the new object has reached a stable state, the constituant objects could be re-established by removing the two arcs introduced earlier. In the object-oriented environment the programming effort required for this implementation is minimal.
Consumer Ib)
1'200
f8 : b
T203
The non-simultaneity constraints on the connecting arcs comes from the fact that one service need will not be assigned more than one resource in a given configuration, whereas a given resource could be assigned to several service needs in the same resource configuration, first because the service needs could be treated sequentially and second. there are potentially more than one configurable element in a resource.
f4
I ••• ·)
T202
The only new element introduced due to the establisment of the communication is the element (2,3). i.e. (fl -f6) . The other elements contain information on the two generic nodes. This representation clearly reflects the fact that these are two communicating agents.
Figure 3.2. Ordinary Petri net structure representing the matching of locally configured elements of a resource and local resource assignments for service needs of a consumer.
P202
T201
Figure 3.4. The incidence matrix of the Petri net of fig. 4.6.
' -_ _ _ _ _ _ _-' (conOgured re.oune - TlOJ elemen •• Ire.dy h."lng .erved _ service nee")
Ruource
T200
3.3. Environment of a node
b;
f10:d~d.
A possible mechanism is illustrated in fig. 3.5 (we first presented these structures in (DEVAPRIYA, 1991a; b» which corresponds to opportunistic problem solving
Hence the incidence matrix of the connected pair of Petri nets is,
259
open nature of our architecture we can benefit from these concepts to further improve communication efficiency.
implemented according to the blackboard paradigm. Here a node calling for the treatement of its local hypothesis modification does not know the node which will respond. Inside a node also its KSs (knowledge sources) are disconnected (uncoupled) and react in an opportunistic manner. This control structure permits parallel activation of KSs. The specification of KSs permits the avoidance of conflictual situations. We see the intervention of the two service nodes (demand-class and task-class) in this communication.
3.4. Distributed blackboards Distributed blackboard systems represent an important research effort to realize real-time response, limited communication bandwidth and reliability in distributed problem solving systems. This model is built as a network of nodes, each of which is a complete blackboard system (like Hearsay-H) (ERMAN, 80). Any given node has direct communication capability with its neighbouring nodes. Any hypothesis modification on the blackboard of a node will be assessed locally for its impact on the activity of the neighbouring nodes. If the impact measure is sufficiently high, then an abstraction of this hypothesis modification would be broadcast by the TRANSMIT knowledge source of the node to all the nodes of its neighbourhood. The RECEIVE knowledge source of a receiving node would integrate the received information onto its own blackboard.
Let us look at fig. 3.5 a little closer. This shows how a partial result posted at a node could invoke KSs of another node. This posting done in the object-part of the node. It could be the pre-condition in a rule of the Implementation Object Control Structure (Imp.OBCS) of this planning node, which has as action part, a service demand. Such a demand consists of a message to the corresponding demand-class sub-node (since the demand-class node is distributed) calling for a service, and hence it is received at an input port of the Spec.OBCS of the node. The role of the Spec.OBCS is to indicate, depending on the state of the objectls of its node, which of its KSs can respond to this message. The termination effect of the possible activation of a KS (according to the corresponding KS specification) is posted on the object/s of the demand-class node. This posting could result in the formulation of a service demand by the Imp.OBCS of the node for the creation of a task instance. Note that there has been no direct service given to the calling node, and hence this is one way communication. When this service demand, in turn, is received at the task-class sub-node (this node controls tasks) it could lead to the creation of the task instance at the object of this node . For the implementation of this task, the Imp.OBCS of the task-class node sends a service demand to the nodes (or node) that specialize(s) on this plannning problem (node 2 in this case). This could be done using message passing or broadcasting.
This mode of problem solving reduces the need for synchronization of information transmission among nodes and the processing order within nodes. This reduction in explicit synchronization has the extremely favourable effect of reducing the response time of the system, because it increases the possibility of parrallel processing. The communication bandwidth could be expected to be reduced due to the possibility to achieve effective internode cooperation using abstract information. In order to increase network reliability, a knowledge based mechanism called murmuring is used for a node to retransmit high-impact information if during a specified time interval it neither receives nor generates higher impact information. Murmuring can be used to correct for lost communications.
We saw that the communication between a service calling node and the demand-class node is one way; from the former to the latter. The communication between the demand-class node and the task-class node is equally one way ; from the former to the latter. On the other hand, the task-class node and the service giving node have a two way communication.
This mecanism is different from that given in fig. 3.5 in that the former uses broadcasting exclusively, whereas the latter is not costrained in this choice (it can use message passing non-simultaneity constraint arcs etc. in addition). SCOPE is intended to provide the facility of dynamic binding the communication mechanism among nodes . This is an important learning function neglected in many of the earlier distributed problem solving architectures.
Yet there are some important problems to be resolved. The data base accesible to service nodes in the structure has to be defined. Do they have access to the databases of all the nodes whose local solutions are matched by them? How does this help to overcome communication bottlenecks?
Given the choice of Petri nets interpreted as objects as the computational model of SCOPE, and the extensive use of pattern matching in this interpretation, we tried to benefit from advances made in the parallel implementation of distributed AI software «DELGAOOFRIAS, 90), (QIU, 90). We presented some results in this direction in (DEV APRIYA, 91c and 92a). Notably we gave a method for implementing high level Petri nets using associative memory.
In fact we have eliminated one souce of communication bottlenecks by distributing the data base and the control among the nodes. We use the structure of the problem to identify a neighbourhood for each node. This is done based on direct access to distributed specializations of the unique nodes. A node that can directly call for services from a demand-class sub-node of a given specialization would have for its neighbourhood, the nodes that can be directly invoked for executing the tasks generated by the task-class sub-node of the same specialization.
With an implementation of the more complex nodes using highly parallel CAM chips, SCOPE could benefit from the flexibility offered by opportunistic KS activation while effectively overcomming risks of communication bottlenecks. It is popssible to implement a class of problem solving nodes that contain KSs whose pre-conditions, trigger-conditions etc. could be periodically matched against the marking of the Petri net objects in the task-class node. This is
Work on distributed blackboards has resulted in the definition of some key notions in inter-node communication (NIl, 89). Without compromising the
260
Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1991b). Petri net based node structures for distributed problem solving in FMS control. to appear in The International Journal on Computer Integrated Manufacturing Systems. Butterworth Publ.. U.K .• 1991. Devapriya, D. S .• B. Descotes-Genon and P. Ladet (1991c). Issues in overcoming communication bottlenecks in an opportunistic distributed problem solving architecture. In the proceedings of The International Workshop on Open Distributed Processing. Berlin. Germany. October 8 -11. 1991. Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1992a). Petri nets interpreted as objects for distributed intelligent control: implementation issues. SICICI '92. Singapore International Conference on Intelligent Control and Instrumentation. Singapore. February 17 - 21 . 1992 . Durfee. E. H.. V. R. Lesser and D. D. Corkill. (1985). Coherent cooperation among communicating problem solvers . In Proc. 1985 Distributed Artificial Intelligence Workshop. pp. 231-276. Dec. 1985. Erman. L. D.• F. Hayes-Roth. V. R. Lesser and D. R. Reddy (1980) . The Hearsay-I! speechunderstanding system: integrating knowledge to resolve uncertainty. Computing Surveys. 12. 213 - 253. 1980. Hewitt. C. and H. Lieberman. (1984). Design issues in parallel architectures for artificial intelligence. Proc. 28th IEEE Computer Soc. Int. ConL. San Francisco. CA. Feb. 1984. pp. 418-423 . Lesser. V. R. and D. D. Corkill. (1981). Functionally accurate. cooperative distributed systems. IEEE Trans. on Systems. Man and Cybernetics, Vol. SMC-11. No.l. January 1981. pp. 81-96. Nii. H. P. (1986). Blackboard systems. blackboard application systems from a knowledge engineering perspective. The AI Magazine. August. 1986. Nii. H. P. (1989). Experiments on Cage and Poligon: measuring the performance of parallel blackboard systems. Distributed Artificial Intel!jgence. Vol.
possible since the KSs are specified using Petri nets and the nodes are implemented as coloured Petri nets and hence their state is represented by their marking. The marking vector of any of these Petri nets can be stored in a CAM word. The parallel matching of an incomming marking vector with the vectors stored in the chip can provide the necessary control for the realtime invocation of Knowledge Sources. With this implementation technique. the nonincrementality of the Petri net token player is no more a constraint for the use of high level Petri nets in realtime control. 4. CONCLUSION Dynamic binding of the communication paradigm between cooperating problem solving nodes in a Distributed Problem Solver is introduced as a measure for overcoming communication bottlenecks in such architectures. This approach is complementary to our earlier propositions on the distribution of control and VLSI based implementation of complex Petri net structures. A new technique is detailed for a type of result sharing cooperation. These results are expected to help in a better adaptation of high level Petri nets as a computational model for the real-time control of Integrated Manufacturing Systems. REFERENCES Davis. R. and R. G . Smith. (1983). Negotiation as a metaphor for distributed problem solving . Artificial Intell .. Vol. 20. pp. 63-109. January 1983. Delgado-Frias. J. G . and W. R. Moore. (1990). A semantic network architecture for knowledge base systems. Eng. Appli. of AI. Vol. 3. March. 1990. pp. 4-10. Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1990) . A Petri net based blackboard type architecture for FMS control (SCOPE). APMS'90. International Conference On Advances In Production Management Systems. IFIP W. G . 5.7. Dipoli. Espoo. Finland. August 1990. Devapriya. D. S .• B. Descotes-Genon and P. Ladet (1991a). Distributed intelligence systems for FMS control using objects modelled with Petri nets (SCOPE Blackboard). IFAC Symp. on Distributed Intelligence Systems. Arlington. Virginia. U.S.A .• August. 1991.
n. Qiu. G. and J. D. Birdwell. (1990). Distributed realtime expert control system. In Proceedings of the Conference on Decision and Control. Honolulu. Hawaii. Dec. 1990. pp. 1525-1530.
NocIol
NocIo2
Figure 3.5. Communicating nodes. 261