Throughput calculation for basic stochastic rendezvous networks

143

Throughput Calculation for Basic Stochastic Rendezvous Networks C.

Murray Woodside

Department of Systems and Computer Engineering, Carleton University, Ottawa, Canada K1S 5B6 Received May 1986 Revised March 1987 and December 1987

Distributed computer programs using rendezvous for intertask communication have performance effects due to tasks waiting for rendezvous. Tasks have a dual nature as being both servers of requests and customers in making requests of other tasks. In a previous paper, such tasks have been termed 'active servers'. In this paper, a distributed program is modelled as a stochastic network of tasks related by their rendezvous requests. For purposes of defining the throughput, the tasks are assumed to have maximum concurrency as if each task were executed on its own processor. For small networks, exact throughput values can be found by translating them into equivalent times Petri nets, and the translation procedure is given as an algorithm. An approximation suitable for larger networks, which was found to be accurate to within a few percent in most of the cases tested, is also given.

Kevwords: Computer Performance, Distributed Software, Parallel Processing, Queueing, Petri Nets, Decomposition, Ada Tasking Performance.

C. Murray Woodside studied at the University of Toronto and at Cambridge University, England, where he received the PH. D. degree in Control Engineering in 1964. Since 1964 he has researched in the fields of control systems, queueing, and performance modelling of both computer and communications systems. His current research interests include performance evaluation and distributed system design methods to meet performance requirements. Since 1970 he has taught at Carleton University, Ottawa, Canada, where he is Director of the Real Time and Distributed Systems Group, and a project Director in the Telecommunications Research Institute of Ontario. He has also spent one-year periods at Imperial College, London, England, and at the University of Paris XI, Orsay, France. During 1983-1984 he was the founding Director of the Ottawa-Carleton Institute for Electrical Engineering. North-Holland Performance Evaluation 9 (1988/89) 143-160

1. Introduction A n e w m o d e l c a l l e d stochastic rendezvous networks is d e f i n e d h e r e a n d u s e d t o c a l c u l a t e t h e throughput concurrent

performance of an important class of systems, namely those using a ren-

dezvous for communications between parallel processes or tasks. The new model differs in several essentials from queueing networks which are commonly used for performance calculations. It has been developed to reflect the special, partly overlapped pattern of execution characteristic of the r e n d e z v o u s , as d e s c r i b e d m o r e p r e c i s e l y b e l o w . T h e rendezvous is a f e a t u r e o f a l a r g e n u m b e r of new computer languages and operating systems w h i c h s u p p o r t c o n c u r r e n c y . I n s o m e o f t h e s e (e.g. C S P [8], T h o t h [1], a n d t h e V - S y s t e m [2]), t h e rendezvous employs message passing using sendand-wait and receive-and-wait primitives. In others (e.g., A d a [3] a n d X M S [4]), it u s e s a p r o c e d u r e - l i k e r e q u e s t / a c c e p t c o n s t r u c t w h i c h is e q u i v a l e n t f r o m t h e p o i n t o f v i e w o f f l o w o f c o n t r o l , a n d w h i c h in distributed systems may also conceal a messagep a s s i n g m e c h a n i s m . T h e p r e s e n t w o r k is p h r a s e d i n t e r m s o f m e s s a g e s a n d a s s u m e s t h a t a ren d e z v o u s is e x e c u t e d as f o l l o w s : (1) T h e t a s k r e q u e s t i n g t h e r e n d e z v o u s s e n d s a message which may include arguments and command information to the accepting task, and waits for a reply. (2) I f t h e a c c e p t e r h a s a l r e a d y e x e c u t e d a rec e i v e i n s t r u c t i o n a n d is b l o c k e d w a i t i n g f o r i n p u t , it n o w r e c e i v e s t h e m e s s a g e s a n d e x e c u t e s a b o d y o f c o d e ' w i t h i n t h e r e n d e z v o u s ' , c a l l e d h e r e its first phase o f execution, a n d s e n d s a r e p l y t o t h e requester. If the accepter has not yet executed ' r e c e i v e ' , t h e n t h e m e s s a g e is q u e u e d u n t i l it d o e s execute a suitable receive instruction. Messages are taken from the queue in FIFO order (although other orders can also be analyzed). (3) T h e r e q u e s t e r is r e l e a s e d f r o m t h e r e n dezvous by the reply and may resume. The accepter may also continue and execute further phases:

0166-5316/89/$3.50 © 1989, Elsevier Science Publishers B.V. (North-Holland)

144

C.M. Woodside / Basic stochastic rendezvous networks

these ' post-rendezvous' phases are concurrent with the requester. A program that is written using tasks as the unit of concurrency and rendezvous (to be abbreviated RNVs) for intertask communications embodies a network of RNVs. This paper describes a Stochastic Rendezvous Network (SRVN) as a model for a class of these programs having a simplified and uniform stochastic structure. Tasks request RNVs with other tasks randomly and independently, according to defined probabilities. Requests are queued and served according to some discipline; F I F O order will be assumed below. The literature on performance modelling of concurrent systems is large, and only a few representative papers will be mentioned here. Standard queueing theory does not model parallel execution with interdependencies between customers, but has been adapted by Heidelberger and Trivedi [7] to a limited variety of splits and joins in the control flow. A variety of methods have been used to model interference between independent tasks due to shared data (e.g., [6]) and there has been extensive modelling of special concurrency control algorithms for obtaining and releasing locks efficiently; a recent example can be found in [5]. Petri nets can model arbitrary interactions among parallel activities and, with the addition of timed transitions and queues, have been used for performance modelling. Holliday and Vernon [9] describe a Petri-net-like modelling framework which leads to a Markov chain performance model. Kant [11] describes a slightly different Petri-net-based model. Radia [15] gives a direct derivation of a Markov chain model for tasks communicating by nonblocking message-passing, while the present author [18] used an approximate decomposition approach to model similar tasks with blocking (rendezvous) communication. Among these previous studies, only the ones described in [18] and [11] treat delays due to RNVs. 1.1. The basic stochastic rendezvous network

This paper describes a general class of multitasking systems with interdependent tasks, interacting (and possibly delaying each other) through RNVs. The basic SRVN is defined as a collection of tasks which: execute independently with stochastic, independent, identically distributed (i.i.d.) execution

times of known distribution, different for each execution phase, - execute in a cycle of two or more phases, the first of which is 'in rendezvous' between accepting a R N V and releasing its caller, and -request RNVs with other tasks, randomly selected with known probabilities, which depend on the phase. A formal graph notation for the SRVN is given in Section 2 with the name 'communication graph'. The unrestricted pattern of interaction between tasks means that the SRVN can represent, in some degree, all the above situations. In the basic SRVN, each task offers its own parallel service as if it had a private processor; this gives the m a x i m u m possible concurrency for the given structure of tasks and interactions. With this limitation, the SRVN can represent the performance behaviour of any program written with RNVs. Processor contention is easily added to the model following the pattern used in [18]. In [18], a limited preliminary version of the rendezvous network was proposed under the name of 'Active Server Networks'. The tasks are active servers in that they both demand service and give it; this is the key difference between SRVNs and queueing networks. The preliminary version permitted only a single reference task compared to multiple reference tasks defined below; the R N V request graph had to be strictly acyclic and the number of phases of a task had to be just two; these restrictions have been lifted. The present work also presents throughput bounds based on service times and relative workload demands, a procedure for deriving an equivalent timed Petri net from a basic SRVN, and an improved approximate solution algorithm for larger nets. In the sequel we define some notation (Section 2) and then address the structure of the R N V communications graph and its effect on the workload of the system (Section 3). Bounds on throughputs are derived in Section 4. In Section 5 we describe the construction of a stochastic Petri net model and its use for calculating throughputs of small SRVNs. For larger networks, an approximate implicit decomposition calculation is described in Section 6, with comparisons of exact and approximate results for small examples. The basic SRVN will be described and analyzed without reference to computers, tasks or devices, as an abstract set of servers doing work and

145

c.M. Woodside / Basic stochastic rendezvous networks

i n t e r c o n n e c t e d b y a stochastic p a t t e r n of R N V s . W h e n the servers are tasks in a c o m p u t e r - s y s t e m design, the m o d e l t h r o u g h p u t c o r r e s p o n d s to the case where each task can execute on its own, which is the best t h r o u g h p u t o b t a i n a b l e u n d e r a n y division of tasks a m o n g processors where there m i g h t be a d d i t i o n a l c o m p e t i t i o n a m o n g tasks that are r e a d y - t o - r u n , for an a v a i l a b l e processor.

an R N V consists of a message (the request, with i n p u t data), a ' f i r s t service phase', a n d a reply. O n a c c e p t i n g an R N V server S i enters its first service phase, d u r i n g which it a l t e r n a t e s service p e r i o d s of its own with requests m a d e to o t h e r servers for nested R N V s . Its own service p e r i o d s are r a n d o m with an e x p o n e n t i a l d i s t r i b u t i o n a n d m e a n rate ~,1- A t the e n d of each of these services periods, it selects server Sj a n d m a k e s an R N V request with p r o b a b i l i t y ~r,j1 or, with p r o b a b i l i t y ~r,01, it t e r m i n a t e s the first phase. A t this point, the original requester of the R N V a c c e p t e d b y S, is released to c o n t i n u e w h a t e v e r it was doing, a n d a sequence of later p h a s e s of server S, begins. T h e later p h a s e s p = 2, 3 . . . . . P, execute similarly to p h a s e 1, with c o r r e s p o n d i n g p a r a m e t e r s tt,t,, ~ r , , a n d

2. Server behaviour in the basic stochastic rendezvous network (SRVN) I n f o r m a l l y , a server in a basic S R V N b e h a v e s as follows. It accepts R N V s from all requesters, in the o r d e r in which the requests are m a d e , where

ServerResource return

P ~

request

Server modue l

~

~

(accept)

~

Phase-2module

from nested RNVs)

' from (return nested RNVs)

~j ,I phase I)

Yjkl

{

,L -I

-

(requests for nested RNVs in Phase I)

(Surrogate delays)

Yjk2 (requests for nested RNVs in Phase 2)

(Surrogate delays)

Fig. la. Petri net (GSPN) module for a two-phase server with nested rendezvous. Phases 3, 4..... P/ are similar to phase 2.

146

C.M. Woodside

/ Basic stochastic rendezvous networks

Server Module f o r Sj I

Ph~ase ~

Phase 3

Phase

P.

=( )

3

,

I

~j2F

~

Phase

yjk2./ .

.

.

.

I .

L

I i i

l v

J

nested RNVs o f Phase 2

~

j

nested RNVs of Phase 3

nested RNVs of Phase Pj

Fig. lb. Petri net module (in the outer dashed box) for a nonaccepting server with phases 2, 3..... py.

~ri0e for phase p. Pi may be any number from 1 up; at the end of phase P~, server S~ is ready to accept another RNV. It should be noted that the second and later phases are different from the first in that Si executes in parallel with its requester. The phases after the second are used optionally to describe a sequence of separate activities with distinct values of ~ and ~r; the use of more than two phases is optional for modelling convenience. The stochastic assumptions of the basic SRVN are: independence of the selection of the next event (nested R N V or termination) after each service period, and independent random service times with separate exponential distributions for each phase. The approximate algorithm in Section 6 also accommodates nonexponential service times. These stochastic assumptions are intended to represent a great variety of task behaviours in an approximate way. If we examine the possible behaviours closely we find that the SRVN models some of them only loosely. Consider a task con-

trolling a semaphore, and a user task executing a specified workload between the granting of the semaphore and its release. The task which controls the semaphore is blocked between the request R N V and the release RNV, although it does no work and is not busy in the interval. In using SRVNs we assume that there is no particular behaviour pattern of this sort dominating the system, so that independent stochastic RNVs and the resulting analysis give a reasonable approximation to real network behaviour and performance. A more formal model of a typical server is defined by the Petri net model in Fig. la. Petri nets in their basic form (see, for instance, [14] for a description) are widely used for modelling concurrency and synchronization. In a Petri net there are places (circles) which hold tokens and transitions (bars). A transition absorbs one token on each input arc (there must be one on each for the transition to fire) and emits one on each output arc. Thus, at the transition labelled 'end phase 1'

C.M. Woodside / Basic stochasticrendezvousnetworks

Server Resource Place

RNV

I

-1 Phase-2 module

'1 II

L

Fig. lc. Petri net ( G S P N ) f o r a server w i t h n o n e s t e d R N V s .

the token executing phase 1 is absorbed and two are emitted. One token returns to reactivate the requesting server, the other goes on to execute phase 2 of Sj. There are various ways of introducing time and mechanisms for choice, among which the Generalized Stochastic Petri Net (GSPN) formulation due to Marsan, Balbo and Conti [13] was used here because it matches the stochastic assumptions about service times and random path selection in our model. Fig. l a represents the sequence of operations of a two-phase server Sj and its interactions with a server S~ which requests an RNV, and with a set of servers (represented by S k and St). The execution cycle of server ~ can be traced in the figure from the resource place through phase 1 (execution and nested RNVs) and subsequent phases (only phase 2 is shown), and back to the resource place. The token (i) in Fig. l a represents the requesting server and its phase, token Sj the accepting server, and the circles labelled ' p h l serv'

147

and 'ph2 serv' are places which hold a token during the service intervals of server Sj. These intervals are controlled by the timed transitions (the rectangles) labelled with the service rates /~jl and ~j2 respectively. The places labelled 'choose' hold the token for zero time while it chooses among the instantaneous transitions (the bars) labelled with their relative frequencies Yjkp (proportional to the probabilities ~rjkp). Each instantaneous transition selects a nested RNV with another server, so the token leaves the subnetwork shown and enters another one as a requester. At this point, a place and a timed transition are shown in the figure, labelled 'surrogate delays' (see [19] for a discussion of surrogate delays in queueing networks). It stands for the total delay in the nested RNV, with mean wjk sec. This is the average time for both waiting and phase-1 execution at server S k, before the return to the server shown in the figure. The surrogate delay wjk is only an approximation to the real delay, because it is assumed to have an exponential distribution. The Petri nets for two special types of servers are shown in Figs. lb and lc. Fig. lb shows a server which does not accept any requests, but cycles in an infinite loop. Such servers may occur as system drivers; we term them nonaccepting servers, and define them to have no first phase but an endless cycle through phases 2,3 . . . . . ~ . Fig. lc shows a second special case with no nested RNVs and only a phase 1, typical of a simple 'remote procedure call' style of interaction. The workload of server S, is usefully summarized by three quantities derived from the descriptions above:

Yijp = mean number of requests by S, for RNV with Sj, during phase p of S, = qTijp(eB'Op) - 1 ,

~i

=

Zp' p = 1Yijp

= mean number of requests for RNV from S i to Sj, per initiation of Si,

Sip = the total service by S~ itself in phase p, in seconds per RNV accepted

= (ttip%op) I = (1 + Ejyop)lxTp l, w~j = mean total delay to Sg between requesting an R N V with Sj and being released at the end of its phase 1. This delay is to be found by the analysis.

148

C.M. Woodside / Basic stochastic rendezvous networks

I n the G S P N model of a task, the relative frequencies Yjkp m a y be used as path selection parameters instead of the probabilities ~rjkp, with ~rj0p replaced by 1, as in Figs. la, lb, and lc.

3. Stochastic rendezvous network (SRVN) structures T h e c o m m u n i c a t i o n pattern a m o n g servers is described by an S R V N c o m m u n i c a t i o n s graph, shown in Fig. 2 for several examples. It is a directed graph with a node S i for each server and a directed arc (Si, Sj) for each pair of servers with Y,j > 0. The arc is labelled with (y~j~ . . . . . YijP,), as illustrated in Fig. 2(a). There are two important types of c o m m u n i c a t i o n s graph, acyclic and cyclic.

)

3.1. A cyclic graphs Acyclic S R V N s are important because of possible R N V deadlocks. Each server requesting an R N V waits for the acceptor, so cycles in the graph represent potential waiting cycles (see, e.g., [10] for a discussion of deadlocks and mutual waiting). Since an acyclic S R V N does not have these waiting cycles, it cannot have deadlocks due to RNVs. In an acyclic graph, the servers (nodes) will be renumbered (again without loss of generality) so that every directed arc (Si, Sj) has i < j , as in Figs. 2(b) and 2(c). Fig. 2(b) shows a typical acyclic SRVN, which will be used in later examples. The reference task S 1 loops endlessly and calls S 5 on average once, S: twice, and S 3 five times per repetition. The first phases of these three called tasks are serialized

,o)

(0,1) (1,0) ~ (1,0) ~'~ e

d Fig. 2. (a) Si requests an RNV with 5 (b) Acyclic RNV graph with one reference task (R = 1). This is the principal example (e.g., Table 1 in Section 5). (c) Acyclic SRVN with two reference tasks (R = 2). (d) A stable-cyclic SRVN with systolic loop S1- - S 2 - - S 3. (e) A convergent-cyclic SRVN.

149

C.M. Woodside / Basic stochastic rendezvous networks

because S 1 is blocked for each one, but their second phases execute in parallel. Contention will occur at several points, for instance if S 1 makes a request for S 2 while it is busy, or if S 2 and S 3 conflict in sending to S4.

tern, ignore the effect of the deadlock avoidance mechanism on the performance. A more advanced SRVN, or a more detailed model such as the Petri nets discussed below, would be needed to capture the avoidance mechanism.

3.2. Cyclic graphs 4. Throughput relationships and bounds An SRVN with a cycle in it can be rescued from eventual deadlock in various ways, either by timing out all requests so that the requester continues with some other operation, or by controlling the cycle by a protocol executed by the servers themselves. Such protocols may be designed using a control token, as described by Le Lann for data access control [12]. We shall analyse cyclic SRVNs approximately under the assumption that deadlock is prevented but without explicitly modelling the mechanism of prevention. In cyclic SRVNs there are three possibilities to consider: (1) Dwergent cycles. There is an explosion of R N V requests around a cycle, making a response infinitely long in time, and violating the assumption of a statistical steady state. Graphically, this condition gives at least one cycle with the loop product Y,jYjk -.- > 1. We shall assume that this is not the case. (2) Stable cycles. There are no divergent cycles in the graph but there are one or more cycles with a loop product of exactly unity, and thus with probability one of repeating the cycle. This case includes self-driven systems timed by a deterministic ring of tasks or systohc loop, as in Fig. l(d). There may be more than one of these driving cycles. (3) Convergent cycles. Here, all cycles have loop products less than unity. A sufficient condition that a phase-1 request cannot cause a request for the same task while it is still blocked, and thus cause deadlock, is that Y~jl = 0 for all the arcs (S~,Sj) in the cycle. These three possibilities are more formally defined in the following section, in the determination of a matrix a of relative server throughputs. The remainder of the paper mostly considers SRVNs with acyclic graphs. Acyclic, stable cyclic, or convergent cyclic cases may be analyzed by the algorithm of Section 6. The performance results for a basic SRVN with a cyclic graph must, because of the assumed stochastic interaction pat-

The basic SRVN is self-driven in an endless loop, or with a set of interacting loops. These loops are driven by a set of one or more nonaccepting servers which do not accept RNVs but rather cycle endlessly through their phases of service. After completing phase P,, they recommence at phase 1. They are not completely freerunning, however, since they may have to wait for the release from the RNVs they request. These nonaccepting servers act as controllers or sources of work for the system so their throughputs, in complete cycles (phase 1, 2 . . . . . P, for server Si) per unit time, will define the system throughputs. For this reason they will also be termed reference servers, and without loss of generality they will be renumbered 1, 2 . . . . . R. The throughputs for the reference servers 1, 2 . . . . . R are special, and are termed reference throughputs. Define f, = rate of requests for server S,, or cycle rate for a nonaccepting server = throughput of server S i. It is assumed throughout this paper that the SRVN is in a statistical steady state, so that the rate of requests for each server equals the rate of completion of service. Therefore, the throughputs of the task are interrelated by the synchronisation imposed by the RNVs, which is expressed by the following traffic equations. For nonreference servers ~ , j > R, we set the rate fj of cycling equal to the rate at which requests to SJ are generated by other servers, to give N

= E

P,

N

E y , j , = Z v,jL,

i=1 p=l

j=R+I

i=1

..... f.

(1)

This set of equations can be transformed, for example by Gaussian elimination, to the following form which determines fR+l ..... fu as a linear

C.M. Woodside / Basic stochastic rendezvous networks

150

combination of the reference throughputs fl . . . . . fn, with coefficients air: R

fi = Y~ airf, ,

i=R+I

. . . . . N.

(2)

r=l

The coefficients air are linear weights on the throughputs, found by the Gaussian elimination process. Implicitly along with equation (2) we also have art = 1,

r = 1 , 2 . . . . . R.

The server throughputs in the SRVN are tightly synchronized by equation (2), and much of the SRVN topology and relationships are captured in the a matrix. The air coefficients are roughly analogous to the throughput ratios in queueing network models, and the reference throughputs fl to fR are analogous to routing-chain throughputs. The analogy does not carry very far, however. One cannot trace a single token representing a job or task or user, around the network from queue to queue, with transitions governed by routing probabilities and a Markov chain (sometimes called the ' j u m p chain' of the queueing model). The underlying Markovian model here is most easily approached via the stochastic Petri net model described above and in Section 5, and has states which represent ' m a r k ings', or combinations of the states of separate tasks. The different cases of the communications graph will now be revisited in the light of the a matrix: (1) Acyclic graph: the coefficients air in equation (2) can be found by setting a,r = 1 for r = 1, 2 , . . . , R, and then starting at the reference nodes and proceeding down the directed arcs using air = Ejaj, Yji at each node S i for which all the predecessors in the graph have been determined. (2) Graph with a divergent cycle: after Gaussian elimination there is no nonnegative solution for a (i.e., no solution that has all a~r>~ 0), and no feasible set of steady-state throughputs. This case is outside our scope. (3) Graph with stable cycles: The coefficient matrix in equation (1) is of defective rank in this case, so the solution for f is analogous to equation (2) but in terms of C + R independent throughputs rather than just R. Each of the C extra independent throughputs represents an independent cycle. That is, there is a solution analogous to

equation (2), for fR+c+l ..... fu in terms of the first R (nonaccepting) tasks and also C further reference tasks, one related to each independent cycle, with throughputs fR + 1. . . . . fR + c. Therefore, these networks are driven by a combination of R free-running nonaccepting tasks and C free-running cycles of tasks. (4) Graph with convergent cycles: A nonnegative matrix a is obtained and used in the same way as for the acyclic case.

4.1. Throughput bounds for basic SR VNs Zero-order bounds on throughput are of two types. Type 1 or no-contention bounds are found by ignoring waiting. Type 2 or utilization bounds are found from the fact that no single server can have a utilization of more than 100%. Both types of bounds can be found for an SRVN, using the definitions:

Xip = the time to complete phase p of server i including contention. (Notice that xia is seen by requesters to server S, as a service time.) x , ; = the mean total execution time, ignoring con-

tention, to complete phase p of S i. It ignores waiting for nested RNVs but includes the execution time in nested RNVs:

xi-p = ~_,YijpX~ + Sip.

(3)

J

z i = the time to complete all phases of server Si, which is its busy time per request ~pXip

zT= the mean total execution time by all servers required to complete all phases of S i, ignoring waiting: 21- ~ E X~p.

Clearly, it must be true that Xip ~__~XTp and therefore z i >~ zi-. Each reference task cycles with period z r, so fr = z~-1. If we define f + = (zT) -1, it follows that f-1 = z~z 7 =(K)

-I ,

C.M. Woodside / Basic stochastic rendezvous networks

from which fr~
r=l,2

..... R

(type-1 b o u n d s ) .

(4a)

The type-2 b o u n d s originate with the utilization constraint for server S, which can be expressed as

f,z, <~1.0. Thus,

(zi)-'

(z.)

1

In conjunction with equation (2), this translates into a set of linear constraints on the reference throughputs, as follows:

R

E a,rf,<~ ( Z Z ) -1, r=l i = R + 1 . . . . . N (type-2 b o u n d s ) .

(4b)

These bounds apply equally to cyclic and acyclic cases. In an acyclic network, equation (3) can be solved recursively from i = N back to i = 1, while in a cyclic network it provides a set of simultaneous equations for x - .

5. Exact performance calculations using Petri nets To obtain exact performance figures in simple cases, a complete G S P N model for an entire system was constructed using as building blocks the server submodels described in Section 2 and in Figs. la, lb, and lc. A computational tool for G S P N was used to obtain numerical results. H o w ever, the experience with the tool suggests that the Petri net a p p r o a c h imposes a serious limitation on the size of system that can be modelled, and emphasizes the usefulness of the S R V N approxim a t i o n of Section 6. A systematic procedure for translating an S R V N into Petri net form is given below. The parts of Figs. la, lb, and l c inside the dashed rectangles are used as templates for the servers; the choice of template depends on the way the server is connected to other servers in the network. Each request arc in the c o m m u n i c a t i o n graph is replaced by the pair of Petri net arcs labelled 'request' and ' r e t u r n ' in the G S P N template. Each server Sj has single place which is initially marked with a resource token for that server.

151

The key problem in constructing the system Petri net is to make sure that a token which branches out at the 'choose' point of server S,, phase p, returns to the return point of the same server and phase. With coloured Petri nets, the return path could be made dependent on the type (colour) of the token, but we shall not use typed tokens. Also, if there are several different requesters and return points, an ordered queue must be modelled. Fig. 3 shows a queueing structure that holds up to three request tokens and routes each back to its proper destination after service, using special control tokens. W h e n a request token arrives at Sj from S i, phase p, it is entered into the sequence of places on the left (beginning at place Q11) and also a control token is spawned and entered at place Cip1. There is a control-token path for each of the three different return points. As the task token advances in the leftmost queue, its control token advances with it so when it leaves service after phase 1, the control token enables the appropriate return path to server S, phase p. Unfortunately, the n u m b e r of control-token places required is the square of the n u m b e r of return paths.

Procedure for Constructing a Petri Net ( G S P N ) Model from an Aeyclie S R V N Step 1. Construct a subnetwork modelled on either Fig. la, lb, or l c for each server, depending on whether or not it makes and accepts R N V requests. Step 2. F o r each nonzero value of y, jp there is an arc from the requesting server subnetwork choice point to the ' w a i t ' place of S 1. If this is the only arc connecting to Sj, attach it to 'wait', and connect the return path, otherwise leave it unattached. Step 3. For each server with more than one request arc (say k), construct a queue with k control queues and k places in each control queue as well as the main queue, on the model of Fig. 3. Attach the input arcs and return arcs as indicated in Fig. 3. Fig. 4 shows the structure of a G S P N model for the six-server S R V N of Fig. 2(b) m a d e by following these steps, to give a network with 56 places and 50 transitions. In c o m p u t i n g the model throughput for Fig. 4, the G S P N tool first generated the space of all reachable markings (2752 in

152

C.M. Woodside/Basicstochasticrendezvousnetworks from S i, phase p from (i',p')

(i,p)

2

return path to S, l,phase p

(to i',p')

from (i" ,p")

(to i",p")

Fig. 3. Petri net model for a queue at Sj for three sources. Server Sj is indicated without nested RNVs, but they can be included as a straightforward generalization.

n u m b e r ) . A m o n g these, the set of 472 so-called ' t a n g i b l e ' m a r k i n g s was found. T h e M a r k o v c h a i n for which these are the states was then solved numerically. F i n a l l y , the m o d e l t h r o u g h p u t s were f o u n d f r o m the state p r o b a b i l i t i e s . In this case, the p r o b a b i l i t y that $6 is b u s y was used. Since S 6 gives 2a16 time units of service in each response, we f i n d f l = (2a~6 l ) P r ° b [ S 6 t o k e n is n o t idle].

F o r the p a r a m e t e r values of the e x a m p l e in Fig. 2(b), al6 = 5.0. S o m e s a m p l e results are given in T a b l e 1, c o l u m n 3. E a c h result t o o k a b o u t 3 minutes on a SUN-3 workstation. A n a l t e r n a t i v e Petri net which does n o t q u e u e the tokens, b u t serves t h e m in r a n d o m order, is shown in Fig. 5. It was s o m e w h a t smaller (35 places a n d 38 transitions). T o give correct r e t u r n paths, the S 4 a n d S 5 service s u b n e t w o r k s were r e p l i c a t e d in Fig. 5 wherever they were called.

C.M. Woodside / Basic stochastic rendezooas networks

153

YI51

S2

subnet

S3

(as Fig. ] a)

$6 subn(

subnet

I

(as Fig.l a)

as Fig.] C

Y362 Y251

,s

Y342

notw.hq . . . . (as Fig. 3

)

,

,

,,

]

]

]

s. s netw thq . . . . ]

(as Fig. 3

)

Fig. 4. Petri net model (structural skeleton) for the example of Fig. lb, with FIFO order of service.

The results in Table 1, column 3 are exact for the SRVN of Fig. 2(b). The throughputs in the table show the effect of bottlenecks introduced at any server S: which is heavily used (large a jl) or which has a large service time (small #j?). The type-1 throughput bounds from equation (3) are presented in column 2 for comparison. They are closely approached within these results in only three cases, for ~51 = 0.03, #41 = 0.03, and #41 = 0.01. Apparently, it requires very substantial unbalance among the task times and frequencies to drive the throughputs to the bottleneck. Column 4 of the table shows for comparison the exact throughputs for the model of Fig. 5 in which the order-of-service was random. They are

almost exactly the same. The remaining columns in the table refer to the approximation described in the following section. It might appear that the SRVN model is unnecessary, since the Petri net can both represent and solve the system. However, in practice the SRVN notation is valuable because - it is more compact and readable, - for large networks, which are impractical for the GSPN, it leads to the approximate solution methods of the next section. Indeed, the GSPN solver is limited to quite small systems because the number of places explodes (especially the places needed to model the queues), and it becomes necessary to solve a large Markov

C.M. Woodside / Basic stochastic rendezvous networks

154

Table 1 E x a m p l e t h r o u g h p u t c o m p u t a t i o n s . T h e S R V N g r a p h is s h o w n in Fig. 2(b), a n d t h e s t a n d a r d p a r a m e t e r v a l u e s are: o ]1,p = 1.0 f o r all i, p e x c e p t ]111 = 0, • Y121 = 2.0, Y131 = 5.0, Y151 = 1.0, )'242 = 3.0, ):251 = 2.0, Y252 = 1.0, Y341 = 1.0, Y362 F r o m the values o f y , a11 = 1 . 0 , a21 = 2.0, a31 = 5.0, a , u = 11.0, a s ] = 7.0, a61 = 5.0

:

1.0 ( a n d all o t h e r Yijp = 0).

Parameters (differences from defaults)

Zero order bound from e q u a t i o n s (4a), (4b)

Exact GSPN computation ( S e c t i o n 5)

GSPN with random order o f service

Approximate algorithm ( S e c t i o n 6)

Approximation error

Default Ylsl = 3.0 Ylsl = 10.0 Y151 = 30.0 Y242 = 1.0 )'242 10.0 )'242 = 30.0 )341 3.0 Y341 = 10.0 Y341 = 30.0 Ps1 = P 5 2 = ]151 = ]152 = ]151 = ]152 = ]151 = ]1s2 = ]141 = ] 1 4 2 = ]141 = / 2 " 4 2 = ]141 = ]142 = ]141 fi 42

0.0286 0.0256 0.0189 0.01075 0.0286 0.0179 0.00735 0.0182 0.00769 0.00308 0.0316 0.0241 0.00714 0.00214 0.0316 0.0136 0.00454 0,00136

0.0175 0.0166 0.0137 0.00851 0.0196 0.0117 0.00557 0.0122 0.00592 0.00239 0.0192 0.0124 0.00618 0.00210 0.0208 0.0101 0.00425 0.00135

0.0173 0.0164 0.0136 0.00845 0.0194 0.0116 0.00557 0.0122 0.00590 0.00238 0.0191 0.0123 0.00618 0.00211 0.0205 0.0101 0.00425 0.00135

0.0186 0.0172 0.0137 0.00825 0.0210 0.0121 0.00556 0.0128 0.00604 0.00240 0.0202 0.0130 0.00626 0.00184 0.0222 0.00981 0.00382 0.00120

6% 6% 0 - 3% 8% 3% 0 5% 2% 0 6% 5% 1% - 12% 7% - 3% -- 10% -- 11%

=

=

=

=

3.0 0.3 0.1 0.03 3.0 0.3 0.1 0,03

chain. When the queues are not modelled, as in Fig. 5, the replication of subnets that is required to provide correct return paths still causes a state explosion. The typical size of the set of all markings limits the solver to Petri nets with about 64 places and 64 transitions in the implementation used by the author. For cyclic SRVNs, the deadlock avoidance mechanism discussed in Section 2, which must be implemented in practice, must also be included in the Petri net model in some way, in addition to the elements representing the SRVN structure. Otherwise, the Petri net deadlocks and shows zero throughput.

6. T h e ' i m p l i c i t d e c o m p o s i t i o n '

approximation

To permit solution of larger networks with more than a few servers, an approximate solution algorithm is presented in this section. It uses Little's result and heuristic relationships among mean values in the system to compute the throughputs of both cyclic and acyclic SRVNs. Compared to the previous implicit decomposition algorithm in [18], this one permits:

servers with internal service time as well as nested RNVs, ignored previously, - a more accurate approximation to the waiting for a RNV, - multiple reference tasks in the SRVN describing the workload (i.e., R > 1), allowing more complex networks to be solved, - more than two phases of service (i.e., P~ > 2 for any i), - SRVNs with stable or convergent cycles as described in Section 2, as well as acyclic networks. The algorithm determines the reference throughputs fl . . . . . fR from the amount,of time the reference servers S t . . . . . S n take to cycle through their phases. To find this, the mean total service time X,e for each phase p of each server S, is determined. This total includes the delay for waiting and service for all its nested RNVs, so the mean delay wij suffered by server S i in obtaining an R N V with server Sj is also determined for every R N V combination. Then, throughputs are found using the reference task phase service times: -

fr = 1

E

X,e,

r = 1 , 2 . . . . . R,

(5a)

i = R + 1 ..... U.

(5b)

p=l R

fi = • r=l

aiJ~

C.M. Woodside / Basic stochastic rendezvous networks

155

i i

i!

i

I

i

(Y

,

I

S5

C (

L

[--

© I!ui

li (S4 serves S2 phase 2)

(S5 serves S2 phase l )

(S5 serves S2 phase 2)

(S4 serves S3 phase l )

(S5 serves Sl)

Fig. 5. Petri net (GSPN) for the SRVN of Fig. lb, with random order of service.

The algorithm is based on two views of the system, applied iteratively. In the first view, w is considered to be known and x is found; a little reflection shows that x is given exactly by Xip = Sip q- E Y i j p W i j . J

(6)

In the second view, x is assumed known and w is found by a heuristic approximation, w~j is the delay suffered by S i when it requests an RNV with Sj, due to queueing for Sj and then waiting during the phase-1 service of Sj, during which S~ remains suspended: wij = (waiting/j) + xjl.

(7)

The algorithm iterates between equations (6)

and (7). It is initialized using the bounds found by equation (3), as follows: xip = x ~ ,

(8a)

w;j = x jl.

(8b)

The calculation of the term (waitingij) in equation (7) is the major effort of the algorithm. To find it, server Sj is considered in isolation and the stream of requests by S; is assumed to be Poissonian, conditioned on S; not being already involved in phase 1 of an RNV with Sj. Notice, however, that Sj may be executing a later phase resulting from a prior RNV with S~, when S; makes a second request.

C.M. Woodside / Basic stochastic rendezvous networks

156

Define a representation for the random variable whose mean is x; thus, £jp is a sample value of the random duration of phase p of Sj, ll ij k =

the expected number of requests from any S k (other than S,.) waiting at Sj but not in service, at the moment of the request from

s, (by an equally valid interpretation) the probability that, at the time of the request by Si, S k is blocked for an RNV with S/ but has not begun the RNV because Sj is already busy,

qjp = the utilization of phase p of Sj = the steady-state probability that ~ is executing in phase p = (by the assumption of Poisson arrivals) the probability that Sj is executing in phase p at the moment of the request, for p > 2 (see below for phase 1)

is executing phase p at the moment S~ makes its request). Without conditioning on the identity of the requester, this probability is identified above as qjp,

Prob( ,4ij p )

=

Prob(S/executing phase p )

--- qjp =fjxjp

(unconditional),

(10)

Y] xj~ .

(11a)

(waitingij (1))

= ~_, Prob(Aijp)

Vjp +

p~l

u=p+l

The use of the Mean Residual Life in equation ( l l a ) is similar to its use in Reiser's equation (3.27) in [16] for mean-value analysis of F I F O queue delays when service has nonexponential distributions. Here, the extra complete phases must be added on, however. We shall separate out the cases where Sj is busy in a cycle previously initiated by S i, to account for serial dependence. To do this, we define the events Eij p and E,/ as follows:

Eij p = the event that, at the moment of a message

=f~x+~, = the mean time to completion of phase p from the moment of the request, i.e., from a random instant. This is known in the queueing and renewal-theory literature as the mean residual life (MRL) of .~jp (e.g., [20, Section 5.2]),

from S~ to 5 , the latter is in phase p > 1 consequent to the previous RNV with S~; in this event, S~ has overtaken the consequences of its previous call, E,j = the union over p of E~jp. With this notation,

The term (waiting,/) in equation (7) has two components: (waitingij(1)) = mean waiting for the current phase (if any) to complete, and (waiting~j(2)) = waiting for other servers which are ahead of S~ in requesting Sj. (waitingij) = (waitingij (!)) + (waitingij (2)).

(9) To find (waitingiy(1)), suppose that S/is executing phase p when i sends its message. Denote this event as A~jp. The mean remaining time until j accepts its next RNV is vjp + F.PJ u = p + l Xju" NOW consider the probability of event Aijp (i.e., that Sj

Prob(Aij p) = Prob(Aijp [ Eij p ) Prob( Eij ;) + Prob(Aigp [ffSijp )

×(1-Prob(Eijp)). Since Egjp implies Aij p and the intersection of F-Jijp and A i j p is the same as the intersection of Eij and A~jp, this can be rewritten as Prob(Aig p) = Prob(Eij p ) + Prob( Aig p [ ff~ij )(1 - Prob( El~ )). (11b) Now consider Prob(Gjp ). Clearly, S~ cannot find Sj in phase 1 due to itself, because of the blocking in the RNV. Thus, Prob( g i j 1 ) = O.

C.M. Woodside / Basic stochastic rendezvous networks

157

For p > 1, the probabilistic coupling between successive actions of the same requester prompts the following approximation. (Including this adjustment also improved the accuracy of the approximation by a few percent in many cases.) The period from a reply from Sj to S, until the next request from Si to Sj is a race between the next request, and the completion of the processing which follows the previous one. Define

sen among those satisfying condition kT,r we find

7 = the random delay from the reply to the following request; clearly, its mean t is given by t = ( f , ~ j ) - I _ wo"

(waitingi/(1))

Prob( Aup I E O) P r o b ( j active for k =~ i in phase p ) 1 - P r o b ( j active for i)

Thus, combining (lla)-(lle) we obtain

= [Prob(E,/p ) + (1 - Prob( E,j))

Then, the result of the race is, for phase p, P r o b ( Eij p ) =

Prob( 7 < Yj2 + " "

+ xj, ),

which depends on the distributions of [ and £j2 . . . . . x~p. For calculation purposes, exponential distributions were assumed; then, the distribution is entirely specified by the mean (which we know), and for phase 2: Prob( E02 ) = x j J ( x , 2

+ t)

For phase 2, the memoryless property of the exponential distribution gives P r o b ( E o 3 ) = (1 - Hij2)Prob(Eij31ff~i,2) = (1

-

Hij2)xj3

/(x,, + = (1

-

%+

xj..

(12)

u=p+ l

Finally, the mean residual life (MRL) vjp of phase p of a server Sj will now be determined. The phase is made up of a series of intervals, alternating service by @ with nested RNV delays. There are two cases: Case I. The residual begins with internal service by Sj. Because the internal service time is exponentially distributed, it can easily be shown that the remaining phase time is statistically identical to a full phase time; there is no memory of the part of the phase that has been completed previously. Thus we have the conditional expectation:

its own service :) vjp = xjp.

FIo2)Hij 3.

Similarly, for later phases we find Prob(Eijp) = (1 - Hij2)(1 - Hij3) (llc)

and clearly we can add these to obtain P r o b ( E , , ) = )-'. Prob( U, jp ).

(",)

(Conditioned on Sj initially giving

w,,)

• .. (1 - r I i j , p _ l ) I I i j p ,

x

(lld)

P

The conditional probability Prob(Aop[Eij ) is the probability of S i finding Sj in phase p but not as a consequence of a previous message from S,. Assuming this is a random inspection point cho-

Case 2. The remaining phase time begins during a nested RNV by ~ with some server--call it S k. Then, the first interval is the residual of the delay experienced by Sj at S k. But this is just the M R L of the waiting #jk, which was replaced for computational purposes by its mean value w oAfter this first interval, an internal service begins and the remainder has the distribution of a full phase service. Thus, (Conditioned on Sj initially blocked i n R N V w i t h S k : ) vw = w j k + xjp. By the law of total probability, vj is a weighted combination of these cases weighted by their

C.M. Woodside / Basic stochastic rendezvous networks

158

probability. The probability of each condition was taken as the mean total duration of the condition per phase of service, divided by the mean total phase time Xjp. Thus,

+ E(w,k +

n~j,=0,

k

which (using equation (6)) reduces to 2

(13)

This completes the determination of (waitingij(1)). To find (waiting,j(2)) we sum the complete phase service times for which Si must wait at ~ , due to all the requests queued ahead of it at Sj. But, on average there are n,j k requests from Sk, so (waiting~j(2)) =

(14) k

P r o b ( S k is in the queue at Sj) = f k Y k j ( W k j -- %1)" Server S k can only be in the queue at Sj when Sj is doing one of three things: (a) Sj is in any phase after serving some S t (14= i, k). Let the probability be Pa; at a random instant we find

E I,#i,k

k=i.

(15)

The conditioning is important in cases where Pc >> Pa + Pb, which happens if S~ in phase 1 is the major load on ~ , apart from S k in phase 1. Notice that if S~ and S k are the only servers which send to ~ , then Pa = 0 and only Pb, depending on the later phases ( p > 1), appears in the numerator. With this expression, (waitingo) and thus wij can be calculated provided values are known for x, q and those components of w for servers with which Sj has RNVs. In an acycfic network, w can be calculated in a sweep from the bottom of the network to the top; in a cyclic network, some w components are used from a previous iteration.

p

n gives the mean queue populations faced by S~ at the time of its request, and is found by applying Little's result to the values of f , w, and x from the previous iteration. The conditioning on the fact that these are the values seen by S~ makes nij k = 0 for k = i. For k 4= i, the conditional values are found as follows. At a random instant such as an arrival instant, we find (by multiplying the arrival rate of S k at Sj by the queue residence time (wkj - xjl)) that

P,,=

nijk = f k Y k j ( w k j -- x j l ) ( Pa + P h ) / ( Pa + Pb + P,,), k~i,

1 x,.)[y,.wjx,.],

Ojp = Xjp + E Y j k p W j k / X j p . k

the ratio of partial task utilizations of task j, as follows:

ftYtj E xjp • p=l

(b) Sj is in any phase after phase 1, after serving S k or Si. Let the probability be Pb: e,

Basic SRVN Throughput Algorithm. An outline of the algorithm is given here using the equations defined above, organized into the correct sequence for execution. Step 1. Initialize Yijp, t~ip from the problem data. If the R N V network is acyclic as discussed in Section 3, then renumber the tasks so that Yop > 0 only for i < j (this can always be done). Step 2. Determine the coefficients ai, (i = R + 1 . . . . . N, r = 1, 2 . . . . . R ) as follows: (a) if the R N V network is acyclic, then solve equations (1) for a sequentially in reverse order in i, f r o m i = N d o w n t o i = R + l ; (b) otherwise, apply Gaussian elimination to equations (1) to reduce them to the form of equation (2). The a~, are then the entries in the resulting coefficient matrix. Step 3. Initialize the mean phase service times X~p and waiting times w~j (i, j = 1, 2 . . . . . N, p = 1, 2 , . . . , P/) using (8), and initialize f~ ( i = 1, 2 , . . . , N ) using (5). Step 4: Iteration body. Determine: (a) f, from (5a) and (5b) for (i = 1, 2 , N ) . (b) n~jk (i, j, k = 1, 2 . . . . . N ) using (15). (c) Vjp ( j = l , 2 . . . . . N, p = 1 , 2 Pi) from (13). (d) for i = 1, 2 . . . . . N, • w U from (7), (9), (11), (12), and (14) ( j = l , 2 . . . . . N), • Xip from (6) ( p = 1, 2 . . . . . Pi). . . . . .

p=2

(c) Sj is serving S~ in phase 1, for which the probability is Pc = fiYijxjl. In case (c), S/ cannot make a request to Sj, so we condition nij , on (a) and (b) but not (c), using

159

C.M. Woodside / Basic stochastic rendezvous networks

S t e p 5. Termination test: repeat from Step 4 if the sum of relative absolute changes in the components of w exceeds a small threshold value; else go to Step 6 to terminate. S t e p 6. The approximate throughput of class r (reference server r) is fr (r = 1, 2 . . . . . R).

To improve convergence behaviour, underrelaxation was applied at the point where w was updated at Step 4(d), equation (7). Then, convergence was always obtained within 50 iterations, and often within a dozen iterations. The form of underrelaxation used was, to first apply Step 4(d) to obtain a value we shall call wnew, and then to combine it with the value from the previous iteration (call it wprev). The combination is w = wprev + p a r a m ( wnew - wpreo).

This was done for each component of w separately. The parameter value p a r a m = 0.5 was used, giving

Xll=

x21 = 3

i

~

'

~

(2 ,o)

3

~

x =

(.2,.4)

Fig. 6. An example SRVN with two reference tasks.

Task Task

1 throughput 2 throughput

Simulation ( + 99% confidence interval)

Approximation

Error

0.0412 + 0.003 0.0513 5:0.003

0.0388 0.0490

- 6% - 4~2%

w = 0.5(wprev + w n e w ) . Approximation errors which underestimate some waiting time value may propagate upwards to underestimate some reference task cycle times, and thus overestimate throughputs. Because of proportionality (2), this will cause other throughputs to be overestimated and, as a final result, the model may overstate the utilization of some task, even over 100%. This is only a logical, not a physical utilization because of the concurrency of senders and receivers in a rendezvous. The accuracy of the algorithm is investigated in Table 1. Column 4 shows the approximation to compare with the exact G S P N result of column 3, with the percentage error shown in column 5: the example is again the six-server model of Fig. 2(b). The accuracy is mostly better than 10%. Another example with two reference tasks is shown in Fig. 6: the errors (shown with the figure) are in the 5-6% range. Dozens of other examples we have worked show comparable error behaviour. The order of the computational complexity of most of the above steps is N 2 or less but, in Step 4(d), the complexity of finding wij is of order N 3 p (there is also a term N 2 p 2 which would dominate only if P > N; this possibility will be ignored). In the case of a cyclic net, Step 2(b) with Gaussian elimination must be applied to get the

relative messaging rates; this is of order N 3. Thus, the order of the overall time complexity per iteration step is N 3 p . The space complexity is dominated by the N 2 components of w and of the coefficient matrix used in the elimination of Step 2(b), the N 2 p components of y, the N R components of a and the N P components of x. The N3 components of n are not stored, but are generated as they are needed. In practice, y and w are usually mostly zero, since a server communicates with only a few others; let this n u m b e r be bounded by J. Although R and J are much less than N, they may well be of the same order as N. Assuming this, and the use of sparse storage, and noting that P is bounded, the space complexity is thus of order N 2 both in acyclic cases (without Step 2(b)) and in cyclic cases (with Step 2(b)).

7. Conclusions

The approximate algorithm for the throughput of networks of communicating tasks using rendezvous shows adequate accuracy on cases which are small enough to have feasible exact solutions. Experience indicates that it converges without difficulty. It has been applied to practical systems of up to 80 tasks without problems.

160

C.M. Woodside / Basic stochastic rendezvous networks

F o r c o m p u t e r systems modelling, the S R V N m o d e l represents the potential p e r f o r m a n c e of a task structure with task c o n t e n t i o n c o n s t r a i n t s b u t n o processor c o n t e n t i o n . I n this form it is suitable for m o d e l l i n g designs before the tasks are allocated to processors. T o model the p e r f o r m a n c e of i m p l e m e n t a t i o n s , further work is needed to i n t r o d u c e processors which execute each server's ' o w n service' intervals, a n d other multiprocessor h a r d w a r e features. T h e a p p r o a c h used in [18], where processors are modelled as servers receiving a n d q u e u e i n g requests for processing, m a y be used here also.

Acknowledgment J o h n Neilson, Jerzy Miernik, Elias Hagos a n d D o r i n a Petriu m a d e helpful suggestions d u r i n g the p r e p a r a t i o n of the paper; Mrs. Petriu p r o g r a m m e d the final version of the algorithm a n d r a n the simulations. This work was s u p p o r t e d b y the N a t ural Sciences a n d E n g i n e e r i n g Research C o u n c i l of C a n a d a u n d e r its O p e r a t i n g grants a n d the U n i v e r s i t y - I n d u s t r y research program. Part of the work was carried out while the a u t h o r was a visitor at the I S E M Laboratory, U n i v e r s i t y of Paris-Sud, France.

References [1] D.R. Cheriton, The Thoth Operating System: Multi-Process Structuring and Portability (Elsevier/North-Holland, New York, 1982). [2] D. Cheriton and W. Zwaenpol, The distributed V-kernel and its performance for disk.less workstations, A CM Operating Systems Rev. 17 (5) (1983) 129-140. [3] Department of Defense of the United States, Reference Manual for the ADA Programming Language, MIL-STD1815a, 1983. [4] N.D. Gammage and L.M. Casey, The software architecture of a distributed processing system, Proc. 4th Internat. Conf. on Distributed Computing Systems, (May 1984) 414-431. [5] A. Gravey and A. Dupuis, Performance evaluation of two

mutual exclusion distributed protocols via markovian modelling, in: B. Sarikaya and G.V. Bochmann, eds., Protocol Specification,

Testing,

and

Verification,

V1

(North-Holland, Amsterdam, 1987) 335-348. [6] A. Hac, Modelling parallel access to shared resources by queueing networks, Preprint of Internat. Workshop on Modelling and Performance Evaluation of Parallel Systems

Grenoble (December 1984) 40-54. [7] P. Heidelberger and K.S. Trivedi, Queueing network models for programs with internal concurrency, 1EEE Trans. Comput. C-32 (1) (1983) 1099-1109. [8] C.A.R. Hoare, Communicating Sequential Processes (Prentice-Hall, Englewood Cliffs, NJ, 1985). [9] M.A. Holliday and M.K. Vernon, A generalized timed Petri net model for performance analysis, 1EEE Trans. Software Engineering SE-13 (12) (1987) 1297-1310. [10] R.C. Holt, Some deadlock properties of computer systems, Computing Surveys 4 (3) (1972) 169-196. [11] K. Kant, Modelling interprocess communications in distributed programs, Proc. Internat. Workshop of Petri Nets and Performance Models, Madison, WI (August 1987) 75-83. [12] G. Le Lann, Algorithms for distributed data sharing systems which use tickets, Proc. 3rd Berkeley Workshop on Distributed Databases and Computer Networks (August, 1978). [13] M.Ajmone Marsan, G. Balbo and G. Conte, A class of generalized stochastic Petri nets for the performance evaluation of multiprocessor systems, A CM Trans. Cornput. Systems 2 (2) (1984) 93-122. [14] J.L. Peterson, Petri nets, Comput. Surveys 9 (3) (1977) 223-252. [15] S. Radia, Markov model for distributed communicating sequential processes, in: E. Gelenbe, ed., Performance '84 (North-Holland, Amsterdam, 1984) 3-16. [16] M. Reiser, A queueing analysis of computer communication networks with window flow control, IEEE Trans. Commun. COM-27 (1979) 1199-1209. [17] M. Vernon, Performance-Oriented Design of Distributed Systems, Ph.D. Thesis, Univ. of California at Los Angeles, UMI, 1983. [18] C.M. Woodside, E. Neron, E.D.-S. Ho and B.M. Mondoux, An "active server" model for the performance of parallel programs written using rendezvous, Preprint of Internat. Workshop on Modelling and Performance Evaluation of Parallel Systems, Grenoble (December 1984) 157-174; also in: J. Systems & Software 6 (1 & 2) (1986) 125-132.

[19] P.A. Jacobson and E.D. Lazowska, Analyzing queueing networks with simultaneous resource possession, Commun. A C M 25 (2) (1982) 142-151. [20] L. Kleinrock, Queueing Systems, VoL 1." Theory (Wiley, New York, 1975).