Servicing of bursty systems

171

Servicing of Bursty Systems L e o n a r d J. F O R Y S a n d D o n a l d E. S M I T H

I. Introduction

Bell Communications Research, Red Bank, NJ 07733, USA

Abstract. This paper treats the practical problem of engineering an existing system having "bursty" traffic for incremental growth. A typical method of treating such problems is to assume a characterization for the input traffic and then use a network of queues type model to analyze delays and performance. While this approach is appropriate when designing a network, it does not make use of measurements of existing performance. In this paper we will explore a different approach. Using readily available data, we will infer the traffic characteristics at specific points in the network. We will then assume that these characteristics hold when the traffic is increased. We apply this method to a sample packet switch network. The results suggest that the accuracy of this method is surprisingly good. This suggests that the philosophy of inferring traffic characteristics from observed network performance may be extendable to more complex situations that occur in ISDN/BISDN and other applications. Since our work is not fully developed, we detail a number of issues that need to be resolved. Keywords. Packet switching, data network engineering.

Leonard J. Forys received his B.S. degree in E.E. from the University of Notre Dame, 1963, his S.M. in E.E. from M.I.T. in 1965 and his Ph.D. in EECS from University of California, Berkeley in 1968. He was Assistant Professor of EECS at University of California, Berkeley during 1967-68 and joined Bell Telephone Laboratories in 1968. In 1984 he joined Bell Communications Research where he supervises group doing performance analysis of voice and data systems. In the Spring of 1990, he was Professor of Applied Mathematics at the University of Adelaide, Australia. D. Smith received degrees in mathematics at Princeton University (B.A.) and the University of Chicago (Ph.D.). In 1980-81 he was a member of the Institute for Advance Study, Princeton. He was Assistant Professor of Mathematics at Ohio State University from 1981 to 1984. He has worked at Bell Communications Research since 1984 on performance problems arising in operator services and telephone switching. North-Holland Computer Networks and ISDN Systems 20 (1990) 171-177 Elsevier Science Publishers B.V. (North-Holland)

One of the most important measures of the performance of a packet network is the response time, or time it takes for a packet to traverse the network. For the purposes of servicing the network as traffic demands on it grow, it is necessary to have theoretical tools for calculating response times from loads. Such tools are hard to come by, for exact queuing models of packet traffic are analytically intractable. We develop an approximate model for computing response times at a single server, such as a single link or processor. The model captures the burstiness of packet arrivals and the variability of packet lengths. It is designed so that parameters in it are computable from data that should be readily available. It is expected that the model will be used to predict response times after a small change in demand. For example, links in an existing network might be running at 30-40% occupancy and one might want to estimate the performance at higher occupancies (50-60% or more) to gauge the effect of adding a modest number of customers. Under such an increase, it would be assumed that the network topology and routing would remain unchanged. We model arrivals by a batch (or compound) Poisson process. We assume that messages arrive according to a Poisson process and consist of batches of packets. The number of messages in a batch is random. The model is tractable because of the Poisson assumption on batch arrivals; the batches model burstiness. Under the assumption that packets are processed by servers at uniform rates, service times are proportional to packet lengths. Packets are a mix of long (e.g. 128 bytes + overhead) information packets and shorter (e.g. 16 bytes + overhead) acknowledgment packets (plus occasional packets of intermediate length). Thus service times are, for the most part, a mix of short and long times. In particular, they are not exponentially distributed (the analytically preferable choice). A common approach to engineering/analyzing packet networks is to assume that the input traffic to the packet network is known, at least to a few

172

L.J. Forys, D.E. Smith / Servicing of bursty systems

moments. For example, the Queueing Network Analyzer [6] computes approximate response times in a network of G / G / n nodes. It is a much more complicated and more ambitious model than ours. QNA requires as input the mean and variance of both the service times and interarrival times of external arrivals at each node. Then it computes the mean and variance of the interarrival times of the total arrival stream at each node by keeping track of the merging and splitting of traffic streams and of departures. Once the first two moments of the arrival and service processes are found, the network is decomposed and each node is treated in isolation for the purpose of calculating congestion measures like mean delays. Our model treats nodes in isolation from the start. We obtain the parameters we need from measurements of the mean packet arrival rate, occupancy, and mean queue size at a server. A possible compromise between the two methods would be to use our approach to infer the input parameters in QNA. We have not examined this possibility. We have validated our model using a simulation of a packet network created by P.T. Brady of Bellcore [2]. The network simulated is shown in Fig. 5. Our results were surprisingly good. However, we have a number of issues to resolve before we can recommend our approach in general. These are outlined at the end of the paper. 2. The Batch Poisson Model We shall use as our model of finks and nodes in a packet network the following queuing system. A single server serves customers (packets) on a firstcome first-served basis. The service time of a customer is a random variable S with a general distribution. Successive service times are independent and identically distributed. The arrival process is a batch Poisson process, i.e. customers arrive in batches of size B at epochs generated by a Poisson process with rate X. Here, B is a random variable taking values 1, 2, 3 . . . . Our shorthand for a batch Poisson process will be M B. Thus, the system under study in an M S / G / 1 queue. Using results from [1], we can show that the mean queue length for this model is

E ( W ) = ~ P~_ E(S)[C + D/p]

(1)

where, C = ½(1 + Csz) and D = ½(E(B)(1 + C2) - 1). Cs2 is the coefficient of variation of S and C2 is the coefficient of variation of B. Similarly, we can show that the mean number of customers in the queue is

E(Q) = C02 + DO 1 -P

(2)

In what follows, we will concentrate on the mean queue length rather than the delay. But Little's law makes the two quantities equivalent.

3. Applying the Model to Packet Networks One of the main objectives we had in approximating packet traffic by a batch Poisson process was to end up with a model that could predict delays using readily available input data. We shall now describe how we expect to use the model and the data. We shall also describe actual packet traffic to make explicit that which we do not model. Then we shall explain why our model works well despite the discrepancies between it and reality. Our model is to be used primarily for servicing networks. This means that measurements of an existing network will be available and it will be necessary to predict delays for loads slightly higher than those presently handled by the network. For example, one might want to know the delay if a link currently operating at 35% occupancy is loaded to 50% occupancy. We shall make the assumption that as loads grow, both the distribution of packet lengths and the distribution of batch sizes remain constant. This means that the occupancy increases only through growth in X, the arrival rate of messages. This assumption seems reasonable because it means that growth in traffic comes about from an increase in the number of users but the characteristics of traffic generated by each user remain constant. The model requires only the constants C and D for the calculation of mean delays as a function of occupancy. We do not need to know E(B) and CB individually. If an actual network is in operation, we should have measurements of the packet arrival rate XE(B), the occupancy p, and the mean queue size E(Q) for a number of values of p below 0.4, for example. We can then estimate the values of C and D by fitting the curve (2) to these data points.

L.J. Forys, D.E. Smith / Servicing of bursty systems

Then E(Q) can be estimated for higher occupancies. Because the packet arrival rate is known, Little's law can be used to estimate E(W) from

E(Q). The characteristics of traffic in a real packet network are more complex than those of the traffic in our model. Our model neglects some of these and idealizes others. The assumption that arrivals to each server in the network occur in batches is one idealization. Processing at each node smooths out arrivals at later nodes. Arrivals at an access concentrator may indeed come in batches if a user transmits a screen of data. Similarly a packet switch connected directly to a host may receive batches of data. If no other arrivals occur during the processing of the batch of packets, then the batch will be transmitted on an outgoing link in an uninterrupted sequence. However, the packets in the batch will not arrive simultaneously at the other end of the link because it takes time to process and send each packet. Therefore, batch arrivals to packet switches not directly connected to data sources will not occur except by coincidences of packet arrivals on several incoming links. Arrivals could actually be smoother than Poisson. Packets are either full information packets of a fixed length, shorter acknowledgment packets, or, less often, partially full information packets of intermediate length. Since the sequence of interarrival times at an internal node is a superposition of the packet processing and transmission times on the links feeding the node, the coefficient of variation of these times could be less than 1 (the coefficient of variation of exponential interarrival times in the Poisson process). The overall arrival process could therefore be smoother than Poisson. What does it mean to use the batch Poisson model under such circumstances? One answer is that the formula for the mean batch Poisson delay subsumes the M / G / 1 queue length formula as the special case D = 0 (B = 1 with probability 1). Modeling the system by an M / G / 1 queue would amount to approximating the arrival process by a Poisson process. In addition, we implicitly modify the service time distribution to improve the accuracy of the predicted mean queue length. This is so because we obtain the constant C = (1 + C2)/2 in the M / G / 1 queue length formula not from measurements of the service times but indirectly by choosing a value of C that gives the best fit of the queue length curve (2) to measured values of

173

the queue length at different occupancies. Compensating for a apparently mediocre approximation to an arrival process by artificially adjusting the service time distribution has been successfully used in modeling telephone operator systems [5]. Using the batch Poisson model extends this technique, for we compute C and D by a fit of the queue length curve to data points. The batch Poisson model may then be interpreted as M / G / 1 with an extra degree of freedom provided by the parameter D. Therefore, it should be able to predict mean queue lengths even better than the M / G / 1 model. One feature of packet networks absent in our model is window flow control. Window delays increase when loads are high. Should our model be able to account for windows by incorporating their effect into the parameters C and D? The only reason why not is that the two parameters are calculated from mean queue length observations at low occupancies when window delays are small. At higher occupancies, the model could underestimate delays if window delays were to grow too fast with load. In our simulation tests, this was not the case. This should probably be attributed to the robustness of the model. Finally, the model ignores errors in transmission. Retransmission inflates load but only mildly at occupancies in our range of interest. Assuming that errors are always present with the same characteristics (as opposed, for example, to a noisy line that gets fixed), the model should be able to account for the effects of errors on delays.

4. The Simulation The performance of our model was tested using P.T. Brady's simulation [1]. This simulation is of a packet network with three packet switches each connected to three access concentrators, as shown in Fig. 5. It models packet traffic in considerable detail, following the X.25 protocol on layer 3 and the LAPB protocol on layer 2. The link transmission rates are 64 kbit/sec. There is flow control at both layers 2 and 3. In our runs, we used a layer 3 window of size 3 and a layer 2 window of size 7. We set the Dbit equal to zero. Each run lasted 250 seconds. During each run the occupancy, mean queue length, and packet arrival rates were measured.

174

L.J. Forys, D.E. Smith / Servicing of bursty systems

It is possible to model propagation delay in the simulation. Propagation delay represents delay due to repeaters on a long transmission path as well as processing delay in access concentrators or packet switches. It is distinct from and in addition to the time, determined by the line speed, required for the output of a frame by a line handler. In our runs, we had either no propagation delay or a round trip delay of 20 ms. The main effect of propagation delay is to slow down window rotation. The queuing discipline at a link in the simulation is round robin with respect to virtual circuits. This is an idealization of what actually occurs. Except for the effects of window flow control, packets in a packet switch are typically served first in first out. In addition, a packet switch is often composed of a number of processors, which handle groups of inputs. Also, the functions of the processors vary, e.g. one processor only handles layer 2 functions, while another routes and handles billing. Although the simulation has provisions for error generation, detection, and retransmission, our runs were done under error-free conditions.

5. Simulation Results Only some of the simulation runs will be discussed in this paper. In these simulation runs, batch Poisson traffic was offered to the network and the mean queue length at either link 1 or link 10 was measured as a function of occupancy. Also in these cases, PS 3 was inoperative, so ACs 1-3 communicated with ACs 4 - 6 only over link 10. When link 1 was observed, AC 1 transmitted information to AC 4. When link 10 was under study, traffic was sent from ACs 1 - 3 to either AC 4 or uniformly to ACs 4-6. Of course, even when information was transmitted in one direction only, level 3 response packets and level 2 acknowledgment frames were sent in the opposite direction. The simulation showed that the batch Poisson model could predict delays with acceptable accuracy. In all cases, we could fit a curve of the form (2) to data points (occupancy vs. mean queue length) at low occupancies that predicted mean queue lengths to within about 15% at higher occupancies, even much larger than those for which

we expect to use the model to make predictions. We found also that an M / G / 1 model was as accurate as the batch Poisson model on link 10. The reason probably is that by the time traffic reaches link 10, it has been smoothed and is no longer batchy. Therefore, the service time distribution in an M / G / 1 model can be adjusted so that an M / G / 1 queue length formula fits the data. The behavior of the two models on link 1 corroborates this explanation. Arrivals here were genuinely batchy and the M / G / 1 model performed much worse than the batch Poisson model. Despite the batch Poisson model's good predictive ability, there were some problems with calculating values of C and D in (2) to give a good fit to data. One or the other of these constants often turned out to be negative! In terms of physically realizable parameters, this is meaningless. The source of this " a n o m a l y " is in the calculation of the two constants by a least squares curve fit. The calculation is very sensitive. Both C and D can be changed appreciably with little effect on the curve (2). Hence the resulting curve deserves more attention than the individual constants. A more elaborate curve fitting procedure might do away with at least the negative values of C and D, but the discussion in the appendix suggests that anomalies/sensitivities will be present in any procedure. We shall now describe the simulation results in detail. We begin with the tests on link 1. Packets arrived at AC 1 and were sent to AC 4. The arrival process was batch Poisson with batches geometrically distributed of mean size 4. The Poisson arrival rate was varied in order to change the occupancy of link 1. There were 8 simulation runs in which the occupancy of link 1 (in the direction of information transmission) increased from 0.116 to 0.903. Except for statistical problems, and the round robin discipline, we would expect that the batch Poisson model should give good predictions. Figures 1 and 2 show the results of fitting the batch Poisson mean queue length curve (2) and an M / G / 1 mean queue length curve to the second and third data points, i.e. those with occupancies 0.230 and 0.356. It is evident that the batch Poisson curve tracks very well the data points corresponding to higher occupancies and that the M / G / 1 curve fails to do so. This result gives a preliminary indication, that only a few measured data points, of relatively short duration, are re-

L.J. Forys, D.E. Smith / Servicing of bursty systems 40

175

80

/

6O

30

20 O"

: 20 10

o: m

1.0

0.2 0.4 0.6 0.8 Occupancy fit to 2nd and 3rd data points

0.0

-20

oo

i

02

I

o.

016

0'8

10

Occupancy fit to 2nd and 3rd data points

Fig. 3. Link 10, batch Poisson model.

Fig. 1. Link 1, batch Poisson model.

quired to give reasonable accuracy for the batched Poisson model. We tested our model on link 10 under a number of different scenarios. We will discuss the scenario where information was sent in both directions. As before, messages arrived according to a Poisson process and contained a geometrically distributed number of packets. (Scenarios where the batch size was assumed to be binomially distributed produced similar results.) Traffic in the forward direction was varied to produce a range of occupancies on link 10 (in the forward direction). Traffic was transmitted in the reverse direction also; its intensity was held constant so that link 10's occupancy in the reverse direction was about 0.5 when no information was sent in the forward direction. Traffic originated from ACs 1-3 and ACs 4-6. Each AC contributed 1 / 3 of the traffic in one direction. During this test, the

average message length was 8 packets. (Scenarios where the average batch size was 4 produced similar results. We chose to illustrate the larger batch for emphasis.) We used bidirectional traffic in this test for two reasons. First of all, bidirectional traffic is more realistic than traffic consisting of information transmitted in one direction only. Secondly, we wanted to examine whether the assumption in our model that service time distributions do not change with load was a limitation. Service times in the present test did change with load. The constant load in the reverse direction caused a constant level of layer 2 and 3 acknowledgments (short service times) in the forward direction. As the information sent in the forward direction was increased, the proportion of long service times (information packets) in the forward direction on link 10 increased. Thus, the service time distribution became more heavily weighted

120

30

100

25

~' 8o

"~ 20

=~ 6o

315

.E

4o 20 t+

0

0.0

I

+

÷

i

0.2 0.4 0.6 0.8 Occupancy fit to 2nd and 3rd data points

Fig. 2. Link 1, M / G / 1 model.

1.0

0 0.0

i 0.'2 0.4 016 018 Occupancy fit to 2nd and 3rd data points Fig. 4. Link 10, M / G / I model.

1.0

LJ. Forys, D.E. Smith / Servicing of bursty systems

176

validation needs to be performed on actual packet networks, - more realistic packet switch models need to be investigated, the method needs to be extended to handle multiple servers. As we indicated earlier, this paper was meant more for discussion purposes and as an exposition of an approach to the practical engineering of systems having bursty traffic. Although we have concentrated on packet networks, the approach may be applicable to other networks as well. -

Fig. 5 Packet network in Brady simulation.

with long service times as load increased. The results of this test show that our model can absorb at least the load dependent service time distribution just described. The simulation was run 7 times in this test. Figures 3 and 4 compare the batch Poisson and M / G / 1 fits to the second and third data points (occupancies 0.334 and 0.438). These show that the models perform about equally well. Similar conclusions follow from comparisons at other data points. Changing the service time distribution does not seem to have impaired the ability of the models to predict mean queue lengths at higher loads from queue lengths at low loads. We mentioned earlier that the calculation of the constants C and D was very unstable. This same phenomenon occurred in this example. These remarks are made more precise in the appendix.

6. Summary We have shown by means of simulation that the batch Poisson model can be used successfully to predict delays at a single server in a packet network. The model meets several conditions dictated by the application we had in mind. First, the data needed as input by it had to be available directly. Secondly, the model should work not only for Poisson arrivals but for bursty arrivals also. The batch Poisson model was able to predict delays in the latter case but the M / G / 1 model could not. A number of areas still need investigation before the applicability of our approach can be established: - more complex networks need to be examined, more substantial statistical analysis needs to be performed,

Acknowledgment

We would like to thank P.T. Brady for performing m a n y of the simulation runs. Also, we wish to thank C.D. Pack and T. Harris for their useful comments on the paper.

Appendix

In fitting curves to various subsets of data points, we found that C and D varied a lot but different values were always close to a line of slope about - 2 / 3 in the C-D plane and produced curves (2) that were close. This may be accounted for b y considering the function F : (0, 1) X R E ~ R 2 defined by F(x, C, D) = (x, Cf2(x) + Dfa(x)). For a fixed choice (Co, Do) of (C, D), F is just a curve as in (2). If G is such a curve, the inverse image F - ] ( G ) is a surface whose intersection with the plane x = a is the line x = a, aC + D = k(1 - a)/a, where k = Cof2(a) + Dofl(a ). Pictorially, the surface may be generated by starting at (0, C0, Do) with a line of slope 0, movig from this point parallel to the x-axis, and rotating the line clockwise so that it has slope - 1 at x = 1. A neighborhood of the surface then maps to a neighborhood of the curve (2) with parameters Co and D 0. N o w a plane of the form 2/3C + D = const will not lie entirely in such a neighborhood but since the values of (2) are very small for x small, and the number 2 / 3 is near an average of the slopes of the fines making up F-I(G), the plane will m a p to a set close to the curve G. This explains why different choices of C and D near a line of slope - 2 / 3 gave rise to curves that were all close together and so accounts

L.J. Forys, D.E. Smith / Servicing of bursty systems

for the instability/sensitivity of the calculation of C and D.

[3] [4]

References [5] [1] D.P. Gaver, Imbedded Markov chain analysis of a waiting fine process in continuous time, Ann. Math. Statist. 30 (3) (1959) 698-720. [2] P.T. Brady, Performance of a simulated X.25/X.75 packet

[6]

177

network with variations in levels 2 and 3 protocols, unpubfished work. R. Cooper, Introduction to Queueing Theory (North-Holland, Amsterdam, 1981). L. Kleinrock, Queueing Systems, Vol. 1: Theory (Wiley, New York, 1975). D. Sze, A queueing model for telephone operator staffing, ORSA J. (March-April 1984). W. Whitt, The queueing network analyzer, Bell System Tech J. (November 1983).