A communication mechanism supporting actions

97

A Communication Mechanism Supporting Actions Kurt ROTHERMEL IBM Deutschland GmbH, European Networking Center, Tiergartenstr. 15, D-6900 Heidelberg, Fed Rep. Germany

Many systems provide mechanisms for defining groups of logically related operations. For example, atomic transactions are the basic grouping concept in database systems and office activities in office information systems. In this paper, we will represent a mechanism for interprocess communication, which supports grouping of operations and forms a suitable base for developing a wide range of distributed systems, Operations can be grouped into so-called actions, where an action may represent an arbitrary (atomic or non-atomic) unit of work. Since actions are known at this level, rather complex mapping functions can be realized as part of the communication mechanism. It will be shown that providing those functions by the communication mechanism instead of implementing them in the application can improve system efficiency, simplicity, and modularity. Besides introducing the underlying system model, our communication mechanism is described in detail, and several examples demonstrating its use are given. Furthermore, the presented mechanism is compared with other communication schemes described in the literature. Keywords: Data Communications, Distributed Action-Oriented Applications, Requestor/Server Model, Addressing Mechanisms.

Kurt Rothermel studied Computer Science at the University of Stuttgart, Fed. Rep. Germany, from which he received his Diploma in 1981 and his I doctor's degree in Computer Science I in 1985. From 1981 to 1985 he was a research assistent at the Computer Science Department of the University of Stuttgart. From 1986 to 1987 he was a Post-doctoral Fellow at the IBM Almaden Research Center in San Jose (12 months) and the IBM Science and Technology - LILOG Department in Stuttgart (9 months). In 1988 he joined IBM European Networking Center (ENC) in Heidelberg, Fed. Rep. Germany, where he is a Research Staff Member.

North-Holland Computer Networks and ISDN Systems 15 (1988) 97-108

I. Introduction A great n u m b e r of systems offer m e c h a n i s m s for defining a n d i d e n t i f y i n g g r o u p s o f o p e r a t i o n s that are logically related in some way. In those systems, which we will refer to as a c t i o n - o r i e n t e d systems, the work l o a d is p a r t i t i o n e d into identifiable units of work, each of which consists of a collection o f operations. F o r example, in d a t a b a s e systems, a t o m i c t r a n s a c t i o n s [10,12] p r o v i d e the b a s i c m e a n s for defining sets of o p e r a t i o n s that are executed in an a t o m i c m a n n e r . O p e r a t i n g systems s u p p o r t the c o n c e p t of a user session [21], which allows for g r o u p i n g the o p e r a t i o n s issued b y a p a r t i c u l a r user. In office i n f o r m a t i o n systems the work is t y p i c a l l y d o n e b y m e a n s of office tasks or activities [14,32], a n d in C A D systems, the o p e r a t i o n s n e e d e d to design i n d i v i d u a l objects are g r o u p e d into so-called C A D t r a n s a c t i o n s [3,17]. T h o s e units of w o r k are typically associated with c o n t e x t i n f o r m a t i o n , which is m a n i p u l a t e d b y their operations. F o r example, t r a n s a c t i o n s are conn e c t e d with recovery i n f o r m a t i o n to e n a b l e transa c t i o n u n d o a n d redo, a n d for each user session c o n t e x t i n f o r m a t i o n is m a i n t a i n e d for a c c o u n t i n g a n d a u t h o r i z a t i o n purposes. C l i e n t / s e r v e r m o d e l s are a very i m p o r t a n t p a r a d i g m for process i n t e r a c t i o n in d i s t r i b u t e d systems. The c o m m u n i c a t i o n m e c h a n i s m we are going to p r e s e n t in the p a p e r is b a s e d o n a c l i e n t / server m o d e l which defines a generic unit of w o r k called action. A c c o r d i n g to this model, a dist r i b u t e d a c t i o n - o r i e n t e d system ( D A O S ) consists of a s e t of clients a n d servers, which c o o p e r a t e with each other for the p u r p o s e of processing actions, where an a c t i o n m a y r e p r e s e n t a n y g r o u p of operations, such as a transaction, a user session o r an office task. I n our model, each request issued b y a client belongs to a p a r t i c u l a r action, a n d all o p e r a t i o n s specified in a request are perf o r m e d as p a r t of the a c t i o n the request is associa t e d with. T h e c o m p u t a t i o n structure of a server can be characterized b y h o w actions are m a p p e d to the processes i m p l e m e n t i n g the server. In exist-

0169-7552/88/$3.50 © 1988, Elsevier Science Publishers B.V. (North-Holland)

98

K. Rothermel / Communication Mechanism Supporting Actions

ing DAOSs, one can find various kinds of server organizations; e.g. a server may be composed of a fixed number of processes, any of which is able to perform requests of any action, or it may consist of a varying number of processes, each of which is dedicated to a single action. The communication mechanism we will present in this paper is an operating system level mechanism for interprocess communication, which is general and flexible enough to form a good base for developing a wide range of distributed systems. It supports the notion of an action, which depending on the system to be developed - may represent any unit of work, and thus can be used to model all kinds of grouping mechanisms applied in distributed systems. Since actions are known at this level, the action-to-process mappings associated with servers can be realized in the communication mechanism, which leads to several improvements: • Efficiency: These mappings can be implemented more efficiently in the communication mechanism than in the DAOS, which improves the overall performance. • Simplicity: Since a DAOS need not realize these mappings, its implementation becomes simpler and hence less time-consuming. • Modularity: The communication mechanism allows server structures to be changed easily without affecting client programs. Interprocess communication mechanisms can be roughly classified by how communication takes place, either directly or indirectly. In the case of direct communication, the sender of a message must explicitly identify the corresponding receiver process, e.g. the services offered by the V kernel [8] and Tandem's G U A R D I A N [4] are based on direct communication. In the case of indirect communication, processes communicate through message receptacles, which are usually called ports or mailboxes. The concept of a port has been introduced by Balzer [2] and has been implemented in many communication systems, such as M A C H [24] and AMOEBA [23]. In direct communication schemes, services are accessed via processes, while in port-oriented schemes, services are typically associated with ports. The communication mechanism we are going to present is also an indirect scheme, in which, however, services are associated with so-called functional port-classes (fp-classes). From a client's point of view, an fp-class consists

of a set of action-specific entries, where the request of a given action must be sent to the entry of this action. The presented communication concept has been implemented in T O C K (Transaction Oriented Communication Kernel), a communication component developed at the University of Stuttgart. T O C K provides services tailored to the specific communication needs of distributed transactionoriented systems, such as services for the initiation, migration and termination of atomic transactions. In this paper, we will focus on the fp-class concept, whose range of applicability is not limited to transaction-oriented systems but can be used in all systems that perform (atomic or non-atomic) actions of some kind. A comprehensive description of T O C K can be found in [25,26]. The remainder of the paper is structured as follows. The next section describes the model of a DAOS, which introduces the required terminology and defines four basic computation structures. In Section 3, the communication mechanism based on the fp-class concept is presented. Section 4 gives some examples demonstrating how the presented mechanism can be used to realize various kinds of computation structures. In Section 5, we justify our claims concerning system efficiency, simplicity and modularity stated above, and finally, we conclude with a brief summary.

2. System Model In our model, a DAOS consists of a set of sites, which are connected via a communication network. On each site, there exist a number of socalled process clusters, each of which is composed of a set of processes. A process cluster can cooperate with other local process clusters as well as clusters residing on remote sites for the purpose of processing units of work called actions. In the literature, the notion of a process has been given numerous definitions. The definition used in our model is much the same as that given by Brinch Hansen [7]: A process is the execution of an interruptible sequential program and represents the unit of resource allocation, such as the allocation of CPU time, main storage areas and I / O devices. Each process belongs to exactly one process cluster (or cluster for short), which resides entirely

K. Rothermel / Communication Mechanism Supporting Actions

on a single site. From the implementation point of view, a cluster is logically associated with a sequential program that is executed by all processes of this clusters. Consequently, processes belonging to the same cluster are executions of the same program and are running on the same site. From the behavioral point of view, a cluster acts as a server that provides a particular type of service to a number of a priori unknown client processes. In order to provide a service, a process of a server cluster itself may exploit the services provided by other clusters, and thus, processes in server clusters may be clients themselves. In our model, we will assume that each cluster of the system acts as a server. That is, each cluster provides a service, which can be exploited by the processes of the other clusters or "external" clients, such as human users and application processes. In order to replicate services, the same service can be implemented by multiple clusters residing on different sites of the system. Actions represent the basic mechanism for defining and identifying groups of operations. An action is a generic unit of work consisting of a set of operations that logically belong together. Unlike transactions [10,12], actions are not required to be processed atomically, which means that an action may be an arbitrary atomic or non-atomic unit of work. This concept of action is general and flexible enough to be used to model all kinds of grouping mechanisms applied in distributed systems. In order to perform operations of an action, a server cluster may use the services provided by the other clusters of the system. Processes in cooperating clusters communicate with each other by exchanging request and answer messages. A client process requests a service by sending a request message to a server cluster providing the desired service. A server receiving a request performs the operations specified in the message and, if necessary, sends an answer message back to the client. In our model, each request is associated with (or belongs to) a particular action. The operations specified in a received request are executed as part of the action the request is associated with. Each action may be executed by a collection of server clusters, which cooperate with each other for the purpose of action processing. In order to satisfy a request of an action, a process in a server cluster may exploit the services provided by the

99

other clusters of the system. Processes sending requests of an action are called the clients of this action. In principle, each process acting on an action's request may issue further requests for this action, and hence, multiple clients may exist per action. For an action a single "external" client initiating the action and multiple "internal" clients requesting services on behalf of the action might exist, for instance. Moreover, it is possible that several clients of an action request the same service. Now, we will define four computation structures that can frequently be found in existing distributed systems. For classifying these structures, we will denote a process executing an action's request as a worker of this action. The examples given below are taken from the class of transaction-oriented systems, in which an action typically corresponds to an atomic transaction. A server cluster C can be structured as follows: (1) Single-Worker/Multi-Action Cluster: C consists of exactly one process that executes all incoming requests. Consequently, in C there exists only a single worker for each action, and the process in C executes on behalf of multiple actions. The transaction manager of POREL [33] and the data servers of C A M E L O T [27], for instance, are implemented as clusters of this type. (2) Multi-Worker/Multi-Action Cluster: C consists of more than one process, and an incoming request may be executed by any process in C. Consequently, for each action there may exist multiple workers in C, and each process of C may work for multiple actions. Server clusters of this type are implemented in ENCOMPASS [6] and G R A P E V I N E [5]. (3) Multi-Worker~Single-Action Cluster: The cluster type has a dynamic process structure. For every incoming request a new process is created in C, and each of these processes performs only one request and then terminates. Consequently, each request of an action is executed by a different process, and every process in C is a worker of a single action. For example, the servers in XDFS [29] and the A R G U S handlers [20] are organized this way. (4) Single- Worker/Single-Action Cluster: This type of cluster also has a dynamic process structure. For each action requesting C's service an individual process is created in C. This process executes all requests belonging to the action and

100

K. Rothermel / Communication Mechanism Supporting Actions

then terminates. Consequently, for each requesting action there exists exactly one worker in C, and each of these processes is the worker of a single action. The Relational Base Machine of POREL [33] and the servers in D I S T R I B U T E D I N G R E S [28] are two examples for this duster type. For each of the computation structures described above we can find conditions under which it represents the appropriate way to organize a server cluster. Which of these structures is best suited strongly depends on the specific characteristics of the system environment as well as the type of service implemented by the server. Discussing the strengths and weaknesses of these structures is beyond the scope of this paper; the interested reader is referred to the relevant literature (e.g. [13,18,19]).

3. Functional Port Classes

The communication mechanism we are going to present in this section is an indirect scheme in which services are associated with so-called functional port-classes (fp-classes). From a client's point of view, an fp-class consists of a collection of action-specific entries. To request a service on behalf of a given action, a client sends a request message to the action's entry in the fp-class associated with the desired service. From a server's point of view, an fp-class comprises a set of ports, each of which represents one or more entries. Consequently, an action-specific entry of an fpclass can be interpreted as a "virtual" port which is mapped to a "real" port of this class. Port-oriented communication schemes support a one-level indirection: A client sends a request to a port, which is mapped to the set of server processes that are able to remove the request from it. The fp-class mechanism also supports indirect communication, but in contrast to port-oriented concepts, a two-level indirection. Using the fp-dass mechanism, the clients direct their request to action-specific entries. Each entry is mapped to one port, which again is mapped to one or more server processes having receive access to it. Both mappings are invisible to the clients. As will be seen later, this two-level indirection is needed to support in a uniform way the different action-to-process mappings associated with server dusters. After this brief overview, we will describe the

concept of an fp-class in more detail. Since clients and servers see fp-classes from different points of view, we subdivide our description into two parts presenting the client's and the server's viewpoint. 3.1. Client's Point of View

From a client's point of view, an fp-class represents a service access point [15] consisting of a set of action-specific entries. A client working for a given action can access a service by sending a request message to the action's entry in an fp-class associated with this service. Each fp-class has a unique address, and an action-specific entry is identified by a two-part address consisting of the address of the comprising fp-class and the identifier of the corresponding action. For example, in order to access a particular service, which is associated with fp-class F, the clients of action A have to direct their requests to A's entry in F, which is identified by address (F, A). 3.2. Server's Point of View

Each fp-class is owned by one server duster, which is called the owner cluster of this fp-class. Only the processes of an fp-class's owner cluster are allowed to remove messages from the ports residing in this class. Consequently, all requests directed to the entries of an fp-class are performed by the processes in the fp-class's owner cluster, or in other words, the owner cluster implements the service associated with its fp-class. From an owner duster's point of view, an fp-class consists of a set of ports, each of which represents the entry of one or more actions. The mapping from entries to ports is only known by the processes of the owner cluster and can be manipulated by creating and destroying ports. An fp-class may contain the following two types of ports: • Specific-Port: Each specific-port (s-port) is logically associated with one (specific) action. The s-port of an action represents the action's entry in the comprising fp-class. To guarantee that an entry is always represented by a single port, an fp-class may contain at most one s-port per action. An s-port is owned by one process of the fp-class's owner cluster, and only this process may remove messages from it. It exists until it is destroyed by a process, or its owner is removed from the system, whatever happens first.

K. Rothermel / Communication Mechanism Supporting Actions

Generic-Port: Each fp-class contains at most one generic-port (g-port). The g-port of an fpclass (generally) represents the entry of every action for which no s-port exists in the fp-class. It can be shared by all processes of the fp-class owner cluster, and exists until it is removed by a process, or the fp-class is destroyed. Table 1 shows how entries are mapped to ports. If the s-port of an action A exists in an fp-class F, it represents A's entry in F, independent of whether or not a g-port exists in F. If F contains a g-port but no s-port associated with A, then the g-port represents A's entry. No entry of A exists when F neither contains a g-port nor the corresponding s-port. We already mentioned that the mapping from entries to ports may be manipulated by creating and destroying ports in fp-classes. In our communication scheme, the ports of an fp-class can be created either explicitly by the processes in the fp-class's owner-cluster, or implicitly by the fpclass itself. Also processes can be established in two different ways: A process in the owner cluster of an fp-class can be created explicitly by another process or implicitly by the fp-class itself. Considering the way how ports and processes can be created, we can distinguish two types of fp-classes, conservative fp-classes and creative fp-classes. A creative fp-class is creative in the sense that it is able to implicitly create ports as well as processes, whereas conservative fp-classes never create ports nor processes. In the following, we will describe creative and conservative fp-classes in detail, where we will use the notation Msg(F, A) to denote the set of messages sent to the entry of action A in fp-class F. • Conservative FP-Classes: A conservative fp-class F can create neither ports nor processes implicitly. The ports of F can be created only explicitly by the processes of F ' s owner cluster, in which a process can be established explicitly only. The communication mechanism performs

•

10l

the following when a message m ~ Msg(F, A) arrives: - If A's entry exists in F, m is placed into the port representing this entry. - If no entry of A exists in F, m is discarded. • Creative FP-Classes: The ports of a creative fp-class F can be created explicitly by the processes of F ' s owner cluster as well as implicitly by F itself, and a process in the owner cluster of F can be established either explicitly by some other process or implicitly by F. When a message m ~ Msg(F, A) arrives, the following is performed by the communication mechanism: - If the entry of action A exists in F, m is placed into the port representing this entry. If m is put into the g-port of F, in addition, a new process is implicitly created in F ' s owner cluster. This process has (receive) access to the g-port, and hence is able to remove m. - If A's entry does not exist in F, the following is done implicitly: First, an s-port associated with A is created in F, and m is placed into it. Second, a new process is established in F ' s owner cluster, which becomes the owner of the newly created port. Note, since the new port represents A's entry, its owner has exclusive access to all messages of Msg(F, A). As will be seen in the next section, creative fp-classes are suitable to implement multiworker/single-action and single-worker/singleaction servers in a very simple and efficient manner, while conservative fp-classes allow adequate implementations of multi-worker/multiaction and single-worker/multi-action structures. To realize these structures, s-ports never have to be created explicitly by processes. Thus, the question may arise, why explicit port creation is actually needed. However, we will see that this possibility substantially enhances the flexibility of the mechanism. It will be shown how explicit port creation can be used to implement computation structures that are combinations of the structures introduced in Section 2.

Table 1 Mapping entries to ports s-port of A existing non-existing

4. F p - C i a s s e s a s S e r v i c e A c c e s s P o i n t s

g-port existing

non-existing

s-port of A g-port

s-port of A

First, we will formulate two essential requirements that should be fulfilled by server clusters for reasons of system efficiency, simplicity and

102

K. Rothermel / Communication Mechanism Supporting Actions

modularity. Then, we will give some examples that demonstrate how these requirements can be fulfilled by using fp-classes as a means for accessing services. Before formulating these requirements we have to introduce some terminology. A service is logically associated with a set of service-access-points (SAPs) [15], which are used by clients to access the service. For example, SAPs can be represented by processes, ports or fp-classes, depending on the underlying communication concept. Moreover, we define ADR(C, A) to be the address of the SAP being used by the clients of action A to access the service provided by server cluster C. Finally, the term C[T] specifies that server cluster C implements computation structure T.

Requirement RI: For all A1,

A 2 E ACTIONS and for all T ~ S T R U C T U R E S : ADR(C[T], A1) = ADR(C[T], A2). That is, for cluster C there exists one SAP address, which can be used by all actions of the system to access C's service.

Requirement R2: For all A ~ ACTIONS and for all T1, T 2 ~ S T R U C T U R E S : ADR(C[T1], A ) = ADR(C[T2], A). That is, the address interface of server cluster C is independent of its computation structure. Remember, in our model, we allow services to be replicated at several sites of the system. In order to achieve replication a n d / o r location transparency [31], a logical addressing scheme [22], in which a permanently assigned service name denotes one or more SAP addresses, is needed. Logical addressing of services requires some form of mapping table for translation between service names and SAP addresses. For this mapping table several ways of implementation are possible, e.g. the table may be distributed over several sites of the system, or a copy of the complete table may exist at one, several or all sites of the system. Due to simplicity reasons, for the following discussion motivating requirement R1 we will assume that services are not replicated. If R1 is fulfilled, we have a simple one-argument mapping from service names to SAP addresses, i.e. each service name is mapped to a single SAP address, which can be used by all clients of the system independent of the actions they are working for. Otherwise, a two-argument mapping that maps (service name,

action identifier) pairs to SAP addresses is needed. The two-argument mapping is required to ensure that all clients of an action use the same SAP to access a given service. For the one-argument mapping the mapping table is easier to implement and cheaper to maintain than for the two-argument mapping. In the case of the one-argument mapping, the mapping table must only be updated when a service is added to or removed from the system (or a service migrates), while for the twoargument mapping a new reference must be added to the table whenever an action accesses a service the first time. Therefore, maintaining the mapping table for the two-argument mapping may cause a substantial amount of overhead, especially if actions are short-lived and action rates are high. This will be discussed in more detailed in the next section. If requirement R2 is fulfilled, the system becomes robust against modifications of computation structures. The address interface of a server remains unchanged even if the server's computation structure is changed completely, i.e. modifying computation structures does not affect client programs. For example, if more parallelism is needed within a server that is implemented as a single-worker/multi-action cluster, this server can be reimplemented as a single-worker/single-action cluster without changing its address interface. It is important to mention that the requirements stated above are not new; they have been formulated in similar words elsewhere (e.g. see [15,31]). The only difference is that R1 and R2 are tailored to our needs in that they assume the existence of identifiable units of work called actions and are based on a set of concrete computation structures. In direct and port-oriented communication schemes, SAPs are typically represented by processes and message ports, respectively. If our communication scheme is applied, SAPs are represented by fp-classes, and an fp-class is considered to be a SAP consisting of a number of actionspecific entries. In order to request a service, a client of an action must direct a request message to the action's entry in the fp-class representing the SAP of the service. Consequently, to be able to access the service, the client must know the SAP address of the service, which is the address of the fp-class representing the SAP, as well as the action identifier, which can be derived from the action

103

K. Rothermel / Communication Mechanism Supporting Actions

context. It is important to notice that entire fpclasses rather than individual entries are considered to be SAPs. The main advantage of representing SAPs by fp-classes is that the SAP address a client must know to be able to access a service is independent of the action the client works for, while the information needed to specify the corresponding entry is independent of the required

service and is available as soon as a client becomes involved in the action. To demonstrate how fp-classes can be applied, we will sketch four different models of a server cluster C, one for each computation structure defined in Section 2. Figure l(a) shows C modelled as a singleworker/multi-action cluster. The SAP of C's service is represented by conservative fp-class F,

F (conservative)

F (conservative)

r:E(*~

r:E(*)

/' t

© ©...©

© (a) single-worker/multi-action cluster

(b) multi-worker/multi-action cluster

F (creative)

F (creative)

r:E(*)

V r / /

1

~

/'

®®

@

(c) single-worker/single-action cluster

o @...@ (d) multi-worker/single-action cluster

F . . . fp-elass Gp . . . g - p o r t Sp(A) . . . s - p o r t associated with action A r : E ( A ) . . . port r e p r e s e n t s e n t r y of action A r : E ( * ) . . . port r e p r e s e n t s e n t r y of e v e r y action W(A) ... process working for action A W(*) . . . process able to work for e v e r y action Fig. 1. Four server models.

104

K. Rothermel / Communication Mechanism Supporting Actions

which only contains a g-port. Since this port represents the entry of every action, all requests directed to F are placed into it. From there, they are removed by the only process of C. Consequently, this process executes on behalf of multiple actions, and all requests of a given action are performed by this process. In Fig. l(b) multi-worker/multi-action cluster C is sketched. This model of C is very similar to the one described above. The only difference is that C consists of several processes, each of which is able to remove messages from the g-port. Therefore, in this model, each process in C may work for different actions, and the requests of a given action may be performed by multiple processes. Figure l(c) models C as single-worker/singleaction cluster. The SAP of C's service is represented by creative fp-class F, which initially is empty. At the time when the first request of an action, say A, arrives, no entry of A exists in F. Hence, the arrival of this message causes a new s-port and process to be created implicitly. The incoming request is placed into the new s-port, which now represents A's entry. Since not only the first but also all following requests of A are placed in this port, the new process, which is the owner of this port, has exclusive access to all requests of A. Of course, after having performed all requests, the process is no longer needed and thus can be destroyed together with its port. In this cluster model, each process works for one action, and conversely, all requests of a given action are performed by a single process. In Fig. l(d), C is modelled as a multi-worker/ single-action cluster, where the SAP of C's service is represented by creative fp-class F. The only port in F is the g-port, which represents the entry of every action. Whenever a request directed to any entry of F arrives, it is placed into the g-port, which causes a new process to be created in C. The new process removes the request, acts on it as required, and then terminates. Consequently, each process in C works on behalf of a single action, and conversely, each request (of a given action) is performed by a different process. In each of the models described above, the service offered by server cluster C is associated with fp-class F. From the clients' point of view, F is a SAP that consist of a set of action-specific entries and is identified by fp-class address F. Obviously, for the cluster models described above

the following holds: For all A ~ ACTIONS and for all STRUCTURES: ADR(C[T], A ) = F.

T~

That is, if fp-classes are used to represents SAP's both Requirements R1 and R2 are fulfilled for the computation structures introduced in Section 2. The examples given above have not demonstrated the whole flexibility of the concept. In all of these examples, ports are created only implicitly. If, in addition, ports are explicitly created by processes, various combinations between the computation structures introduced in Section 2 can be achieved. In the following, we will give an example for each type of fp-class. Example 1. A combination between a singleworker/single-action and a multi-worker/singleaction server can be modelled by explicitly creating ports in a creative fp-class. If no ports are established explicitly in the creative fp-class depicted in Fig. l(d), each incoming request is executed by another process. If the process receiving the first request of an action, say A, is willing to perform the succeeding requests of A as well, it can explicitly create an s-port representing A's entry, from which it can receive all succeeding requests of A. Note that some synchronization between the clients of an action might be needed to insure that the second request of an action does not arrive before the s-port is created. This kind of explicit port creation allows to realize a new type of server cluster, which executes the requests of one action class as a single-worker/single-action server and the requests of another action class like a multi-worker/single-action server. The cluster type could make sense, for instance, if the requests of one action class are context-sensitive, while the requests of the other class are not. Example 2. Another computation structure, which has been implemented in T A N D E M ' s PATHWAY system [30], for instance, can be interpreted as a combination between a single-worker/ single-action and a multi-worker/multi-action cluster. In contrast to the cluster type described in the previous example, this one consists of a fixed number of processes. All requests of an action are performed by a single process, and each process may work for multiple actions over time. This computation structure can be realized by a cluster

K. Rothermel / Communication Mechanism Supporting Actions

owning a conservative fp-class, which initially contains a g-port only. When the first request of an action, say A, arrives, it is placed into the g-port, which represents A's entry at this point of time. From this port, the request will eventually be removed by some process, which is ready to work for a new action. The process receiving A's first request creates an s-port associated with A, which then represents A's entry. Since all subsequent requests of A are placed into this port, the process has exclusive access to all requests of A. In summary, if fp-classes are used to represent SAP's, requirements R1 and R2 are fulfilled for the basic computation structures introduced in Section 2. Moreover, we have shown that the possibility of creating ports explicitly by processes increases the flexibility of the scheme substantially. By applying explicit port creation, we can model various combinations of the basic cluster types, which also satisfy R1 and R2.

5. Discussions The objective of our discussions is to justify the claims concerning system efficiency, simplicity and modularity stated in Section 1. For this purpose, we will first investigate how the four basic computation structures can be realized with communication schemes described in the literature. Then the results of these investigations will be compared with the corresponding realizations exploiting the fp-class mechanism. A single-worker/multi-action cluster is the most simple server type. Since a server cluster of this type consists of a single process, it can be implemented appropriately with port-oriented as well as direct communication schemes. As discussed by Andrews and Schneider [1] direct schemes are not that well suited for multi-worker/ multi-action clusters, whereas indirect communication mechanisms based on ports or equivalent concepts are. The dynamic process structure of a multi-worker/single-action cluster is appropriately supported by communication mechanisms that trigger the creation of a process whenever a request arrives at a server. Such a mechanism, which is based on the concept of "invocation mailboxes", is provided by StarOS [16]. SNA's Advanced Program-to-Program Communication

105

(APPC) [11] provides similar features. To implement a single-worker/single-action cluster, a mapping that maps each action exploiting the offered service to a dedicated process in the clusters has to be maintained somewhere is the system. To the author's knowledge, the communication mechanism described in this paper is the only one which provides this kind of mapping function under the assumption that for an action there may exist multiple clients requesting the same service. If this kind of action-to-process mapping is not supported by the communication mechanism, it has to be implemented in the DAOS (i.e. in the code sitting on top of the communication mechanism). This task can be accomplished by a component which we will call scheduler. We assume that every server cluster C is logically associated with a scheduler, which maps the actions using C's service to the processes in C, either directly or indirectly via ports. Of course, besides these action-to-process mappings, each of which is defined in the context of a given server cluster, there exists a higher-level mapping that maps services to schedulers. How schedulers are implemented is not important for the following investigations, e.g. there might exist one process per node executing all local schedulers, or an individual process per scheduler. In the following, we will describe two basic schemes to involve schedulers in accessing services. In the first scheme, all requests to be performed by a given server cluster, say C, are sent to the scheduler of C. When receiving a request of an action, the scheduler forwards the request to the action's worker if it already exists in C. Otherwise, it creates a new process before forwarding the request. We will refer to this as a "forwarded requesting" scheme. In the second scheme, the client of an action consults the scheduler of C before requesting C's service the first time. If the action's worker already exists in C, the scheduler returns the corresponding SAP address. Otherwise, it creates a new process before returning the address. After having received the SAP address, the client directs the first and all subsequent requests of the action to this address. Since a client directly sends the requests to the processes actually performing the requests rather than to an intermediate process, we will call this a "direct requesting" scheme. Of course, hybrids of these two schemes are conceivable, e.g. the first request

106

K. Rothermel / Communication Mechanism Supporting Actions

of an action could be issued following a "forwarded requesting" scheme, while all subsequent requests could be requested directly provided the answer to the first request returns the corresponding SAP address. An alternative approach to implement servers, which is based on the concept of process groups, has been suggested in [9]. The V kernel provides mechanisms to define process groups and to multicast messages to the members of an identified group. Following this approach, a server cluster can be modelled as a process group. To request a service for an action, a request message containing the action's identifier is multicast to all processes of the corresponding process group rather than being sent to a single process. Only the process that actually works for the action identified in the message accepts the request, whereas all other ones ignore it. Although this approach has attractive properties for several applications, it has some drawbacks in our context: First, each process of a server cluster has to receive all requests sent to this cluster, filter out the ones which are relevant to it, and discard the others. Second, to guarantee that at most one process is created for each action in the presence of multiple clients requires additional logic, whose complexity is comparable with that of the scheduler logic. The investigations above show that existing communication mechanisms are well suited to support single-worker/multi-action, multi-work e r / multi-action and multi-worker/single-action clusters. The reason for that is that the action-toprocess mappings associated with these three cluster types are simple in the sense that a communication mechanism can realize them without supporting the concept of an action. However, as soon as actions are mapped to dedicated processes, as it happens in single-worker/single-action clusters or in the two cluster types describes at the end of the previous section, implementations become fairly cumbersome. To justify our claims, we will now evaluate "direct requesting" and "forwarded requesting" schemes in terms of system efficiency, simplicity and modularity: • Efficiency: In the case of a "direct requesting" scheme, a single-worker/single-action server offers a different SAP address for every action exploiting its service, i.e. requirement R1 is not fulfilled. Consequently, each client must consult

the corresponding scheduler before sending the first request of an action. Of course, this causes extra message passing as well as delays, none of which is acceptable for high action rates. In contrast, with a "forwarded requesting" scheme R1 can be fulfilled for direct as well as indirect communication mechanisms. However, applying this requesting scheme, we have to pay extra cost for message forwarding and context switches. The intermediate process causes at least one additional context switch per request, which may be a rather expensive operation. The hybrid scheme described above provides better performance because the cost for contextswitching and message forwarding has to be paid for the first request only. In contrast, if fp-classes are used, neither consulting a scheduler nor message forwarding is needed because the scheduler function is part of the communication mechanism. At this level, it can be implemented in a highly efficient manner, especially if the fp-class mechanism is integrated in the operating system kernel. . Simplicity: Both the "direct requesting" as well as the "forwarded requesting" scheme require the implementation of the scheduler function in the DAOS, which adds complexity to the DAOS implementation. • Modularity: In the case of "direct requesting" schemes, requirement R2 cannot be fulfilled if direct and p o r t - o r i e n t e d c o m m u n i c a t i o n schemes are applied, while in the case of "forwarded requesting", R2 is satisfied because the intermediate process can realize an arbitrary action-to-process mapping, which never becomes visible to clients. However, the price for this flexibility is the penalty of extra overhead and complexity, as pointed out above. Let us summarize what we can draw from our discussions. If SAPs are represented by fp-classes, the action-to-process mappings associated with server clusters are implemented as part of the communication mechanism. Realizing them at this level in general leads to less complex and more efficient implementations. Neither R1 nor R2 can be fulfilled for "direct requesting" schemes, while both requirements are met in the case of "forwarded requesting", however, at the cost of extra complexity and overhead.

K. Rothermel / Communication Mechanism Supporting Actions

6. Summary A communication mechanism based on the concept of an fp-class has been presented. This mechanism supports the notion of an action, which provides a general means for grouping and identifying arbitrary units of work. Due to the generality of its grouping concept, the presented mechanism is flexible enough to form a good base for developing a wide range of distributed systems. The fp-class mechanism can be classified as an indirect communication scheme supporting a two-level indirection. If SAPs are represented by fp-classes, every request is sent to an actionspecific entry. Each entry is mapped to a single port, which again is mapped to one or more processes. As a result of this two-level indirection, rather complex mapping functions can be realized as part of the communication mechanism. We have discussed why system efficiency, simplicity and modularity can improve if these mapping functions are removed from the DAOS to the communication mechanism. At present, a first prototype of T O C K is running on Digital E q u i p m e n t C o r p o r a t i o n V A X / V M S systems connected via a local area network at the University of Stuttgart. So far, the entire code of T O C K is executed at the utility level. Future work is to integrate time critical pieces, such as the fp-class mechanism, into the operating system kernel in order to improve performance.

Acknowledgment The author would like to thank the referees for their helpful comments on earlier drafts of this article.

References [1] G.R. Andrews and F.B. Schneider, Concepts and Notations for Concurrent Programming, A CM Computing Surveys 15 (1) (1983). [2] R.M. Balzer, Ports - A Method for Dynamic Interprogram Communication and Job Control, Proc. A F I P S Spring Joint Computer Conference, Vol. 38 (AFIPS Press, Reston, VA, 1971). [3] F. Bancilhon, W. Kim and F.H. Korth, A Model of CAD Transactions, Proc. International Conference on Very Large Data Bases (1985).

107

[4] J.F. Bartlett, A NonStop Kernel, Proc. 8th Symposium on Operating Systems Principles (ACM, SIGOPS, 1981). [5] A.D. Birrell et al., Grapevine: An Exercise in Distributed Computing, Communications of the A C M 15 (4) (1982). [6] A.J. Borr, Transaction Monitoring in ENCOMPASS: Reliable Distributed Transaction Processing, Proc. International Conference on Very Large Data Bases, (1981). [7] P. Brinch Hansen, Distributed Processes: A Concurrent Programming Concept, Communications of the A C M 12 (11) (1978). [8] D.R. Cheriton, The V Kernel: A Software Base for Distributed Systems, 1EEE Software 1 (2) (1984). [9] D.R. Cheriton and W. Zwaenepoel, Distributed Process Groups in the V Kernel, A CM Transactions on Computer Systems 3 (2) (1985). [10] J. Gray, Notes on Database Operating Systems, in: Operating Systems: An Advanced Course, Lecture Notes in Computer Science 60 (Springer, Berlin, 1978). [11] J.P. Gray et al., Advanced Program-to-Program Communication in SNA, I B M Systems Journal 22 (4) (1983). [12] T. Haerder and A. Reuter, Principles of Transaction-Oriented Database Recovery, A CM Computing Surveys 15 (4) (1983). [13] T. Haerder and P. Peinl, Evaluating Multiple Server DBMS in General Purpose Operating Systems, Proc. lOth International Conference on Very Large Data Bases (1984). [14] D. Heimbigner and D. McLeod, A Federated Architecture for Information Management, A CM Transactions on Office Information Systems 3 (3) (1985). [15] Information Processing Systems - Open System Interconnection - Basic Reference Model, ISO/TC79/SC16/ 1S7498, 1983. [16] A.K. Jones, et at., StarOS, a Multiprocessor Operating System for the Support of Task Forces, Proc. 7th Symposium of Operating System Principles (ACM SIGOPS, 1979). [17] W. Kim et al., A Transaction Mechanism for Engineering Design Databases, Proc. lOth International Conference on Very Large Data Bases (1984). [18] B.G. Lindsay et al., Computation and Communication in R*: A Distributed Database Manager, A C M Transactions on Computer Systems 2 (1) (1984). [19] B. Liskov and M. Herlihy, Issues in Process Communication Structure for Distributed Programs, Proc. 3rd Symposium on Reliability in Distributed Software and Database Systems (1983). [20] B. Liskov, The ARGUS Language and System, in: Distributed Systems: Methods and Tools for Specification: An Advanced Course, Lecture Notes in Computer Science 190 (Springer, Berlin, 1984). [21] H. Lorin and H.M. Deitel, Operating Systems, The System Programming Series (Addison-Wesley, Reading, MA, 1981). [22] J.M. McQuillan, Enhanced Message Addressing Capabilities for Computer Networks, Proceedings of the IEEE 66 (11) (1978). [23] S.J. Mullender and A.S. Tanenbaum, The Design of a Capability-Based Distributed Operating System, The Computer Journal 29 (4) (1986). [24] R.F. Rashid, From RIG to Accent to Mach: The Evolution of a Network Operating System, Technical Report, Carnegie-Mellon University, 1986.

108

K. Rothermel / Communication Mechanism Supporting Actions

[25] K. Rothermel, Communication Support for Distributed Database Systems, Proc. GI-NTG Conference on Communication in Distributed Systems (1985). [26] K. Rothermel, Communication Concepts for Distributed Transaction-Oriented Systems (in German), Informatik Fachberichte 145 (Springer, Berlin, 1987). [27] A.Z. Spector, Distributed Transaction Processing and The Camelot System, Technical Report CMU-CS-87-100, Carnegie-Mellon University, 1987. [28] M. Stonebraker and E.J. Neuhold, A Distributed Database Version of INGRES, Proc. 2nd Berkeley Workshop on Distributed Databases and Computer Networks (1977). [29] H. Sturgis et al., Issues in the Design and Use of a Distributed File System, A CM Operating Systems Review 14 (3) (1980).

[30] A Close Look at PATHWAY, Software Education and Design Support, Tandem Documentation SEDS-003, 1982. [31] L. Svobodova, Client/Server Model of Distributed Processing, Proc. GI-NTG Conference on Communication in Distributed Systems (1985). [32] G.R. Thoma et al., A Prototype System for the Electronic Storage and Retrieval of Document Images, A CM Transactions on Office Information Systems 3 (4) (1985). [33] B. Walter and E.J. Neuhold, POREL: A Distributed Database System, in: C. Mohan, ed., Recent Advances in Distributed Database Management (IEEE Press, Silverspring, MD, 1984).