- Email: [email protected]
Access flow control algorithms in broadband networks
broadband
Access flow control
algorithms in broadband networks Ibrahim W Habib and Tarek N Saadawi analyse an access flow control algorithm applied to a network's access nodes
In broadband networks, flow control functions are shifted to the edges of the network and implemented on an endto-end basis. Access flow control is then essential to avoid congestion. In this paper we present and analyse an access flow control algorithm which is applied at the access nodes to the network, specifically at the input voice and video multiplexers. The algorithm uses a control signal proportional to the multiplexer congestion level to throttle the peak bit rate of the input arrival process. Two congestion thresholds (KI, I(2) are used to limit the number of cells in the multiplexer buffers. The threshold levels are a function of the input traffic characteristics. We analyse the resulting quasi Birth-Death queueing process for a video-buffer multiplexer. Some performance analysis parameters such as the blocking probability and the average number of bits per sample are presented. Keywords: broadband networks, access flow control, BirthDeath queueing
Recent advances in fibre optic communications, switching and buffering technologies, and voice and video source coding have made integrated networks a possible reality. In these networks, multimedia information is transmitted using the same transmission links and switching fabrics. The multimedia information expected to be carried by Electrical Engineering Department, City University of New York, City College, New York, NY 10031, USA Paper received: 26 July 1991
these networks covers a wide spectrum of traffic characteristics, ranging from low bit rate data to broadband bit rates suitable for video transmission. Integrated networks are thus designed to be flexible enough to support various traffic characteristic sources. It is required that these networks utilize the available high bandwidth efficiently enough such that different traffic sources (e.g. data, fixed bit rate voice, variable bit rate voice and video, etc. can be supported, and so that their required performance measures, also called Class of Service (COS), are met 1' 2. ATM can thus provide us with the required flexibility and efficiency. However, the gained flexibility in accommodating traffic sources with different characteristics may cause serious congestion problems that will lead to severe buffer overflow, cell loss, and degradation in the required service quality. Flow control is required to avoid congestion and allocate network resources efficiently to each call according to a predefined class of service (COS) that defines performance measures 3-7. Because of the diverse mix of traffic sources with different bandwidth and traffic uncertainties, the network must enforce admission control to protect its resources from users who may not confine themselves to their declared traffic statistics (policing enforcement and bandwidth monitoring). The complexity of the problem arises from the fact that the expected traffic sources are very bursty in nature. Sources differ widely according to their traffic characteristics, which determine their degree of burstiness and auto-correlation. There are several definitions for the burstiness of a source. A simple description is to characterize the source as having three main parameters: peak rate, average rate, and average burst length (in
0140-3664/92/005326-07 © 1992 Butterworth-H einemann Ltd 326
computer communications
broadband seconds), which is the average duration of an active period of transmission at the peak rate. However, this simple description does not capture the effect of correlations between the cells' interarrival times of the superposition arrival process. It is, then, more suitable to use other parameters to characterize the burstiness and correlations of the traffic. For example, variable bit rate video sources can be represented by time domain characteristics such as the coefficient of variation, the autocorrelation function and its distribution. Sources exhibit different traffic patterns according to their type (i.e. video, voice, data, interactive image, etc.). They also produce different patterns according to the coding technique used and to the contents of scenes (in the case of video sources) 8-1°. In this paper, burstiness is defined as the degree of variability of the traffic characteristics from that of the Poisson traffic, which is considered to be a smooth traffic. The proposed flow control algorithm is next presented. Then we apply it to the video multiplexer case. Finally, we discuss possible future work.
FEEDBACK ACCESS FLOW CONTROL ALGOR ITH M A preventive control strategy avoids congestion through a two-fold approach: 1 Allocating network resources (such as bandwidth and buffer space) to each call before accepting it, which is called admission control, and is done during the call setup phase. If the resources are not available, the call is rejected 11-1s 2 During the call progress phase the input arrival characteristics, together with the link traffic conditions, are continuously monitored. The arrival characteristics are controlled to their predetermined values, which were agreed upon during the call setup phase (a traffic enforcement and policing function). Bandwidth assignment rules for each call are thus very important to avoid congestion, which may result from overloading the n e t w o r k 16-21.
It is rather difficult to design an access flow control algorithm which can control the superposition stream of traffic belonging to widely different correlations and burstiness coefficients. Variable bit rate video traffic would generate a very high peak bit rate for a short duration of time, while fixed bit rate voice traffic has a relatively low peak bit rate with slow time changes. In general, the traffic mixture is an 'unbehaved' or an 'unsmooth' one, implying that we cannot smooth it via large buffers for several reasons: 1 Long buffers would imply unacceptable delays. 2 Long buffers accommodating several traffic types require extensive priority scheduling techniques which are not favourable in a fast packet switching network. 3 Long buffers would not, provide any advantage 'in terms of smoothing the traffic', since the expected
vol 15 no 5 june 1992
Multiplexer buffer
[
Source coder i
A
"
K2
K1
_L
j,
/
Server
V Controller
Figure 7.
Multiplexer with feedback control
bursts' duration could still be longer than the buffer size. An efficient flow control algorithm must control the input arrival process upstream the network (i.e. at the input access node). The leaky bucket technique is one example 22. In this paper we present and analyse a flow control algorithm based upon feedback throttling of the arrival process to the input statistical multiplexer. We apply a feedback control signal, proportional to the congestion level of the multiplexer, to the input source coder (see Figure 1). The signal will control the source rate by decreasing the coding rate (number of bits/sample). When another congestion threshold is achieved, a similar action is taken, thus reducing the input rate further. This algorithm is not of the reactive type, since it is applied at the input access node to the network, and its speed is not limited by the propagation delay. Any control action taken will be in time to alleviate the potential congestion. In fact we prevent congestion, since the feedback signal is proportional to the number of cells waiting in the multiplexer buffer. The control signal is transmitted using the ATM out-of-band signalling, hence it can be applied to sources directly connected to the network, as well as those connected through high-speed tANs. There are several advantages to this scheme. First, unlike the leaky bucket scheme it does not allow excess cells to enter the network. It prevents congestion, and greatly reduces the potential of congestion downstream in the network, since it reduces the number of cells per buffer. Second, it is applicable regardless of the type of coder used: it is good for variable bit rate coders as well as fixed bit rate ones. Third, it provides the means for the maximum possible shaping of the input arrival process through decreasing the peak bit rate. Consequently, the bandwidth allocated to the input call can be reduced, and yet the same required COS would be achieved. Also, the statistical multiplexing gain is enhanced, since more sources can be supported for each multiplexer, hence the traffic gets smoother and more easy to control. The scheme can thus be used to provide users with economically priced connections, since there are more users for each connection. The price to be paid is that we may perceive a slight degradation in the quality of the voice or video delivered, but we can prove that this degradation is quite marginal.
327
broadband ACCESS FLOW CONTROL IN VIDEO MULTLPLEXERS Modelling and analysis Figure 2 presents the continuous time Markov chain model of the video source, also known as the phase process. Maglarias et al. 23 presented and analysed this model to match the statistics of the continuous state autoregressive model. The autoregressive model is quite accurate in modelling the video source statistics. However, it is quite complicated if we try to use it in analytical studies. A similar model was used by Huang 24 to analyse the video statistics. Saito et aL 2s used a (J-state) Markov modulated process to study the performance of the video multiplexer. The model represents the arrival rate A(t) by quantizing the bit rate into uniform discrete levels, and the rate variations over time are approximated by a continuous time process with discrete jumps at random Poisson times. Thus the state space (A) of the chain represents a quantization level of the original sampled process, measured in bits/pixel. The (M + 1) states scan the range of the variations. The parameters a and /3 are the transitional rates of jumping from one quantization level to the other. These parameters were evaluated by Maglarias eta/. 23 by fitting them to the average, variance and the autocovariance functions of the original measured data of the source statistics. The results are: A = -CR(0) + -E(XD -
E(ZR) = 3.9/
(la)
M (lb)
1 +
a =3.9-~
(Ic)
where E(Z R) and CR(O) are the average and the variance of the aggregate arrival process from R identical and independent sources. Each source transmits a random process with mean E(A) and autocovariance function C ( t ' ) = C(O)e-3"9r. t" is the source frame number ndivided by a frame rate of 30 frames/s. The autocovariance curve was proved by several authors to follow an exponential fit. The value (3.9) was found to match the variations of this specific video experiment. The number of states M was set to be 10 R. It was found by Maglarias et al. 23 that this value of M yielded reasonable results that were close enough to the measured data. In this paper, all the variables used were normalized to cells/ms. The multiplexer buffer has a fixed buffer length of N
!
Figure 3.
!
i
i
Markov chain model
cells and is fed by the process described by the system of equations (I). Figure 3 shows the two dimension continuous time Markov chain model which describes the system's behaviour. The queue length stochastic process is a Markov process at instants of state changes. Each discrete level arrival to the queue is a Poisson process with exponential service time, with meanp where p =/_/C. C is the link capacity in bits/s, and L is the cell length in bits. The cell fixed service time is replaced by an exponential service time. Li 26 proved that replacing the service fixed time by an exponential time does not affect the queueing process, since the correlation effects of the arrival process dominate those of the service time process. Each state of the phase process iA, (0 < i < M), is therefore the equivalent of i sources, each having a Poisson arrival process at the rate of A cells/ms. The transitional rates between the system states are thus Poisson, of rate iA. Let KI and K2 be the queue lengths at which the flow control is activated. If the queue length reaches the threshold limit KI the rates drop to iB, and at queue length K2 the rates drop to iC. The rates B and C represent the arrival rates after decreasing the number of bits/sample of the source coder. In our analysis, we set those values to be 0.75A and 0.5A respectively. Let the duple {Q iA~}, where (,4 E~A, B, C) denote the number of cells in the queue and the phase of the arrival process, respectively. Then the stochastic equilibrium probability of the system is:
P×,v=Pr. {Q =x, ifi,=y} where (0 < i < M).
for(O
iC < y < i A )
We can write the following equations for the system:
MaPo, o = PPl.o + /~Po,A
A
Figure 2.
328
2A
(M -- 1 ) A
MA
Single video source model (phase process,)
for(x=O,y=O)
(2a)
{(M - i)a + iA + i~}Po, iA = I.IPI,iA + (M - i + 1)aPo, (i - 1)A + (i + 1)/3Po,(i + 1)A for ( l < i < M - 1 )
(2b)
(M• + MA)Po, MA
(2c)
= PPI,MA
+
CZPo,(M- 1)A
computer communications
broadband Ma if
for (1 < x < K1): (3a)
( M a + IJ)P×, o = laPx + 1, o 4" flPx, a
I(M - i ) a + i~ + i A + PlPx, i4 = ( M - i + 1)aPx,(i_ 1)4 + laPx + 1, iB -I- (i + 1)/3Px,(i + 1)A + iAPx - 1, hA, for ( l < i < M -1) (3b) (m~ 4- ~I)Px, MA = + PPx + 1, MB
(4a)
I(M - i ) a + ifl + iB + PIPx, iB = ( M - i + 1)aP×,(i_ 1)B + (i + 1)flPx,(i+ l)B + i B P x _ l,iB, -1)
(4b)
(M/~ +/~)P×, MB = aPx, (M - 1)B + M B P × _ 1, MB (4C)
+ PPx + 1, MC
for(K2+l
( M e + ,u)P×, o = PP× + 1, o + flPx, c
[(M - i ) a + i~ + i C + #tPx, iC = ( M - i + 1)aP×,(i_ 1)c 1)/~Px,(i + 1)c + iCP× _ 1, i o - 1)
+ PPx + 1, ic + (i +
for ( l < i < M (aft Jr t_l)Px, MC
4
.
(~P lh MI~
II I
M
u
(tvl I )(z -/I MI~
p
(M
M# MA
]b 1
H,N) =
(3C)
(Ma +/~)P×. 0 = PPx + 1,0 +/3P×. B
+ , u P x + 1,iC
-(kf - / b &-z q fl
=
a P x , (M _ 1)A 4"- M A P × _ 1,MA
for(K1+1
for ( l < i < M
T(ll
(5b)
- a P x , (M - 1)C "{" M C P x - 1, Mc
(5c)
+ PP× + 1, Mc
S is a diagonal matrix where p is its diagonal element. A is also a diagonal matrix where hA is its diagonal element. The matrices T(KI), T(K2) are the same structure as T(1) with the appropriate scaled )is. Similarly, A ( K I ) , A(K2) are the same as A but with the appropriate scaled Zs at each corresponding threshold levels KI and K2. The above matrices can be of extremely large sizes, yielding numerical difficulties in solving them. For example, if there are I 0 video sources and the buffer size is of 100 cells, then the dimension of matrix Q is 10 000, which is impossible to solve using direct matrix manipulations. We used matrix-geometric techniques, introduced by Neuts 27, to solve the above system. The solution uses an iteration refinement technique which needs to be slightly modified to suit the overload control in our case. The details are not repeated here, and the reader is referred to Neuts 27. Finally, the above equations were solved numerically for Px.v- The blocking probability PB is calculated from: M
for (x = N): (6a)
( M a + P)PN, 0 = flPN, C
PB = ~
PN, iC
(7)
i=0
[(m - i ) a + ifl + i C + P l P × , i c = ( m - i + lbP×,(i- 1)c + PPx + l , i c + (i + 1)~P×,(i+ l ) c + i C P × _ l , i O
for ( l < i < M
-1)
(6b)
( M ~ + P)Px, MC = aPx, (M - 1)C + M C P x - 1, Mc
+ PPx + I, MC
(6C)
The above equations can be written in the matrix form PQ -- 0, where P is the steady state probabilities vector and Q is the transitional rates matrix; then we have: 7((I)
H1)
A 5
Q=
T(KT)
A
5
T(K1)
A(K1) 5
T(K2) S
A(KT) T(K2)
A(K2) 5
T(M
T(0), T(1), T(KT), T(K2), T ( N ) , S, A, A(KT), A(K2) are all square matrices of the dimension M, where:
vol 15 no 5 june 1992
NUMERICAL RESULTS The video source characteristics reported above had an average arrival rate of 3.9 Mbits/s and a peak rate of 11 Mbits/s. We used a buffer length of 20 cells to limit the delay to 50 ps, where the cell length is the ATM standard of 53 bytes and C is 150 Mbits/s. Figure 4 reflects the improvement in the multiplexer performance for a different number of sources. The blocking probability has dropped significantly as a result of applying the flow control technique. As the buffer size is relatively small, the statistical multiplexing gain is not very appreciable. Figures 5 and 6 illustrate this effect more clearly. The trend is clear; as the buffer size is increased, the statistical multiplexing gain becomes more effective. However, as expected, the flow control technique has significantly enhanced the multiplexing gain. Figures 7 and 8 compare the performance of the multiplexer for different flow control thresholds. As the threshold levels are decreased, the performance improves; however, the price will be a slight degradation of image quality. The results reported here confirm our earlier discussions. It proves the imnportance of applying this type of flow control technique to accommodate sources with high peak rates without sacrificing the
329
broadband 0.001
0.0~5 0.01
.*
.'"
/ ,...
~ -.•
o O.
0.001
." • •
.90 190
•-
..
..-'
/
.- .....
/
..-
/
.-"
/ /
/"
,s-
.f'"
•.-
.i
.
0.0~3
.• .
.
.
""'i,
/"
.i./f./
9
/"
/
0.0002
-4.
./"
0.0001 •"
..
/
•
.'"
/
,i"
IE-05
1 E-06
f f /I
/
/-
/../ /./ ./
"'--
.. ,. -....
/.,/
%..
0.0001 -
.,,-
~
I
I
I
I
0.5
0.6
0.7
0.8
0.9
1.0
I 3
5E-05 2
Offered load
Figure4. B/ocking probability versus /oad (buffer size = 20). : no control, '/source; ---: no control, 2 sources; • • • : no control, 4 sources; - - - : w i t h c o n t r o l , 1 source; . . . . . : with control, 2 sources; -----: with
4
Number of sources
Figure 6. Statistical multiplexing gain - with control : buffer size = 10; ---: buffer (utilization = 0.8). s i z e = 15; • • • : buffer s i z e = 2 0
control, 4 sources
0.1 0.03 0.025
0.001 ° ° * ,....-
0.02
°°
.-"
••
:~
0.015
o
1E-05
. . . .iiiiiiiiiiiii ...
•
2Q.
.**
1E-07
O.Ol
.-
t f
/
,..
/
•
/
.
/
..'//// /
O_
0
•
• [
..."
14"
.
/
r':':
133
...
. .... ....,... ..... ....,.
1 E-09
it':
ss: ¢..
1E-11 0.005
Statistical
(utilization = 0.8).
3
multiplexing gain
-
: buffer size = 10; s i z e = '/5; • • • : b u f f e r s i z e = 2 0
no
---:
control buffer
efficiency. In our work we have placed a stringent delay limit of 50 ps. However, we can relax this value to I 0 0 Ms without having a major effect on the delay requirement. Therefore, we can double the buffer size and the statistical multiplexing gain with be more effective• The obvious result of decreasing the blocking probability will have the direct impact of minimizing serious congestion problems• Thus, we can efficiently utilize the valuable
330
..........
I
2
Number of sources
F i g u r e 5.
~-!
1E-13 L . 0.2
0•4
0•6
0.8
1
Offered load
Figure 7. Effect of flow contro/ thresho/d /eve/s, single source (buffer size = 20). : no control; - - - : K1 = 8, K 2 = 14; • • • : K 1 = 6 , K 2 = ' / 2 ; - - - : K 1 =4,/(2=8
network resources such as bandwidth and, in the meantime, provide different users with the required performance• Another impact is that we can accommodate more sources at the same bandwidth when flow control is not used. It had been suggested 11 that for bursty traffic with a high peak to link radio, the non-statistical
computer communications
broadband 0.1 ~-~7~.
0.0001
/
1E-07
/ /
.o
O
/ /
.
/ /,,. /..." /..' , /.,: ]..." / "
1E-10 I 1:"
, ,
1E-13
1E-16 I 0.2
0.4
0.6
q 0.8
Offered load Figure 8. Effect of flow control threshold levels, 4 sources (buffer size = 20). : no control; - - - : K1 = 4, K2 = 8; • • . : K 1 = 6 , K 2 = 1 2 ; - - - : K 1 = 8 , K2 = 14
operation mode could be more effective• We believe that with our proposed flow control algorithm we can utilize the VBR coding techniques to operate within the statistical multiplexing region with high efficiency•
the high frequency components due to sudden or abrupt motion in the image, which cannot be dropped. We have found that this flow control algorithm has several important merits in solving the problems of congestion in integrated networks. Besides preventing congestion, it has a major impact on the problem of bandwidth assignment and enforcement. We have addressed the problem of congestion control, and tried to devise unified hierarchical architecture that reflects the dependance and efforts of various network parameters on the performance• Traffic characteristics, call acceptance decision, bandwidth assignment, link traffic utilization and dynamic routing are some of the parameters that influence the design of an efficient flow control algorithm. We have tried to shed some light to this fact. The problem of flow control in integrated networks needs more work to achieve a novel scheme, if there is any. Our work calls for further studies, and some of the interesting outstanding issues are: 1 Performance analysis study of the bandwidth assignment rule with feedback flow control. 2 Problem of optimizing the statistical multiplexing gain with flow control under buffer size constraints. 3 Merits of applying this flow control algorithm at integrated switching nodes with heterogeneous input traffic, and its effects on switch performance• 4 Study of other traffic shaping techniques to decrease the burstiness and smooth out traffic characteristics.
REFERENCES CONCLUSIONS
AND FUTURE WORK
In this paper we have presented and analysed a flow control algorithm applied to the input access node of an integrated network• It operates on the principle of feedback control• The algorithm has been applied to the video cell multiplexer, where the arrival rate is controlled by reducing the number of bits per sample• When the first congestion level is reached, the rate is decreased from 8 to 6 bits/sample, and when the second congestion level is reached, the rate is decreased to 4 bits/sample. The algorithm throttles the peak arrival rate via a simple technique which does not depend upon particular detailed time dependant characteristics of the arrival process• Because the scheme is applied at the input access node, it is not limited by the propagation delay which is dominant in high-speed networks. Very little control can be done along the transit nodes, so it is essential to smooth down the burstiness of the input traffic to avoid the congestion caused by the formation of long bursts inside the network. The most effective method to decrease the burstiness, and hence avoid congestion, is by throttling the peak arrival rate. Cell dropping is not favourable, since it is difficult to distinguish these cells carrying the high resolution image information, which can be dropped, from those carrying
vol 15 no 5 june 1992
1 Coudreuse, J P, Thomas, A and Servel M, 'ATD techniques: An experimental packet network integrating videocommunication', ISS (1984) 2 Turner, J S and Wyatt, L F 'A packet network architecture for integrated services', Proc. GLOBECOMM (1983) 3 Reiner, H Draft Recommendation I. 12"/, CCITT.SGXVIII.TD•no 49, CCITT, Geneva, Switzerland (1988) 4 De Prycker, M, Plehiers, P, Fastrez, M and Bauwens, J 'Evolution towards a Belgian broadband experiment', (1987) 5 De Prycker, M 'Definition of network options for Belgian ATM broadband experiment', IEEEJ. Selected Areas in Commun., (December 1988) 6 Wernik, M 'Architecture and technology considerations for multimedia broadband communications', Proc. GLOBECOMM (1988) 7 Rider, M 'Protocols for ATM access networks', Proc. GLOBECOMM (1988) 8 Nomura, M, Fujii, T and Ohta, N 'Basic characteristics of variable rate video coding in ATM environment', IEEE J. Selected Areas in Commun., (June 1989) 9 Yasuda, Y, Yasuda, H, Ohla, N and Kishing, F 'Packet video transmission through ATM networks, Proc. GLOBECOMM (1989)
331
broadband 10 Verbiest, W, Pinnoo, L and Voeten, B 'Statistical multiplexing of variable bit rate video sources in ATM networks', Proc. GLOBECOMM (1988) 11 Woodruff, G, Rogers, R and Richards, P'Congestion control framework for high speed integrated packetized transport', Proc. GLOBECOIVlM (1988) 12 Takahashi, T and Hiramalsu, A 'Integrated ATM traffic control by neural networks, ISS (1990) 13 Goleslani, S 'Congestion free transmission of real time traffic in packet networks', Proc. INFOCOMM (1990) 14 Cidon, I, Sohraby, K and Bala, K'Congestion control for high speed packet switched networks', Proc. INFOCOMM (1990) 15 Eckberg, A E, Lucantoni, D and Luan, D'Meetingthe challenge: Congestion and flow control strategies for broadband information transport', Proc. GLOBECOMM (1989) 16 Decina, M, Toniatti, T, Vaccari, P and Verri, L 'Bandwidth assignment and virtual call blocking in ATM networks', Proc. INFOCOMM (1990) 17 Gallasi, G, Rigolio, G and Fratta, L 'ATM: Bandwidth assigment and enforcement policies', Proc. GLOBECOMM (1989) 18 Decina, M and Tonialti, T 'On bandwidth allocation to virtual bursty connections in ATM networks', Proc. /CC (1990) 19 Hui, J Y 'Resource allocation for broadband networks', IEEE J. Selected Areas in Commun., (December, 1988) 20 Ohnishi, H, Okada, T and Noguchi, K 'Flow control schemes and delay-loss tradeoff in ATM networks', IEEE J. Selected Areas in Cornmun., (December 1988)
332
21 Wang, W, Saadawi, T and Aihara, K 'Bandwidth variation and control for ATM networks', Proc. ICC (1990)
22 Sidi, M, Liu, M, Cidon, I and Gopal, I 'Congestion control
through
input
rate
regulation',
Proc.
GLOBECOMM (1989)
23 Maglarias, B, Anastassiou, D, Sen, P, Karlsson, G and Robbins, J 'Performance models of statistical multiplexing in packet video communications' IEEE Trans. Commun., (July 1988) 24 Huang, S 'Modeling and analysis for packet video', Proc. GLOBECOMM (1989) 25 Saito, H, Kawarazaki, M and Yamada, H 'Analysis of statistical multiplexing in ATM transport networks', Proc. ICC (1990) 26 Li, S O 'Overload control in a finite message buffer', IEEE Trans. Commun., (December 1989) 27 Neuts, M F Matrix-geometric solutions in stochastic models: An algorithmic approach, John Hopkins Press, New York (1981) 28 Goodman, D J 'Embedded DPCM forvariable bit rate transmission', IEEE Trans. Commun., (July 1980) 29 Brady, P T 'A Statistical analysis of on-off patterns in 16 conversation', Bell SysL Techn. ]., (January 1968) pp 73-91 30 Sriram, K and Whilt, W 'Characterizing superposition arrival processes in packet multiplexers for voice and data', IEEE J. Selected Areas in Commun., (September 1986) 31 Sriram, K and Lucantoni, D M 'Traffic smoothing effects of bit dropping in a packet voice multiplexer', IEEE Trans. Commun., (July 1989) 32 Gross, D and Harris, C M Fundamentals of Queueing Theory, John Wiley, New York (1985)
computer communications
Access flow control
algorithms in broadband networks Ibrahim W Habib and Tarek N Saadawi analyse an access flow control algorithm applied to a network's access nodes
In broadband networks, flow control functions are shifted to the edges of the network and implemented on an endto-end basis. Access flow control is then essential to avoid congestion. In this paper we present and analyse an access flow control algorithm which is applied at the access nodes to the network, specifically at the input voice and video multiplexers. The algorithm uses a control signal proportional to the multiplexer congestion level to throttle the peak bit rate of the input arrival process. Two congestion thresholds (KI, I(2) are used to limit the number of cells in the multiplexer buffers. The threshold levels are a function of the input traffic characteristics. We analyse the resulting quasi Birth-Death queueing process for a video-buffer multiplexer. Some performance analysis parameters such as the blocking probability and the average number of bits per sample are presented. Keywords: broadband networks, access flow control, BirthDeath queueing
Recent advances in fibre optic communications, switching and buffering technologies, and voice and video source coding have made integrated networks a possible reality. In these networks, multimedia information is transmitted using the same transmission links and switching fabrics. The multimedia information expected to be carried by Electrical Engineering Department, City University of New York, City College, New York, NY 10031, USA Paper received: 26 July 1991
these networks covers a wide spectrum of traffic characteristics, ranging from low bit rate data to broadband bit rates suitable for video transmission. Integrated networks are thus designed to be flexible enough to support various traffic characteristic sources. It is required that these networks utilize the available high bandwidth efficiently enough such that different traffic sources (e.g. data, fixed bit rate voice, variable bit rate voice and video, etc. can be supported, and so that their required performance measures, also called Class of Service (COS), are met 1' 2. ATM can thus provide us with the required flexibility and efficiency. However, the gained flexibility in accommodating traffic sources with different characteristics may cause serious congestion problems that will lead to severe buffer overflow, cell loss, and degradation in the required service quality. Flow control is required to avoid congestion and allocate network resources efficiently to each call according to a predefined class of service (COS) that defines performance measures 3-7. Because of the diverse mix of traffic sources with different bandwidth and traffic uncertainties, the network must enforce admission control to protect its resources from users who may not confine themselves to their declared traffic statistics (policing enforcement and bandwidth monitoring). The complexity of the problem arises from the fact that the expected traffic sources are very bursty in nature. Sources differ widely according to their traffic characteristics, which determine their degree of burstiness and auto-correlation. There are several definitions for the burstiness of a source. A simple description is to characterize the source as having three main parameters: peak rate, average rate, and average burst length (in
0140-3664/92/005326-07 © 1992 Butterworth-H einemann Ltd 326
computer communications
broadband seconds), which is the average duration of an active period of transmission at the peak rate. However, this simple description does not capture the effect of correlations between the cells' interarrival times of the superposition arrival process. It is, then, more suitable to use other parameters to characterize the burstiness and correlations of the traffic. For example, variable bit rate video sources can be represented by time domain characteristics such as the coefficient of variation, the autocorrelation function and its distribution. Sources exhibit different traffic patterns according to their type (i.e. video, voice, data, interactive image, etc.). They also produce different patterns according to the coding technique used and to the contents of scenes (in the case of video sources) 8-1°. In this paper, burstiness is defined as the degree of variability of the traffic characteristics from that of the Poisson traffic, which is considered to be a smooth traffic. The proposed flow control algorithm is next presented. Then we apply it to the video multiplexer case. Finally, we discuss possible future work.
FEEDBACK ACCESS FLOW CONTROL ALGOR ITH M A preventive control strategy avoids congestion through a two-fold approach: 1 Allocating network resources (such as bandwidth and buffer space) to each call before accepting it, which is called admission control, and is done during the call setup phase. If the resources are not available, the call is rejected 11-1s 2 During the call progress phase the input arrival characteristics, together with the link traffic conditions, are continuously monitored. The arrival characteristics are controlled to their predetermined values, which were agreed upon during the call setup phase (a traffic enforcement and policing function). Bandwidth assignment rules for each call are thus very important to avoid congestion, which may result from overloading the n e t w o r k 16-21.
It is rather difficult to design an access flow control algorithm which can control the superposition stream of traffic belonging to widely different correlations and burstiness coefficients. Variable bit rate video traffic would generate a very high peak bit rate for a short duration of time, while fixed bit rate voice traffic has a relatively low peak bit rate with slow time changes. In general, the traffic mixture is an 'unbehaved' or an 'unsmooth' one, implying that we cannot smooth it via large buffers for several reasons: 1 Long buffers would imply unacceptable delays. 2 Long buffers accommodating several traffic types require extensive priority scheduling techniques which are not favourable in a fast packet switching network. 3 Long buffers would not, provide any advantage 'in terms of smoothing the traffic', since the expected
vol 15 no 5 june 1992
Multiplexer buffer
[
Source coder i
A
"
K2
K1
_L
j,
/
Server
V Controller
Figure 7.
Multiplexer with feedback control
bursts' duration could still be longer than the buffer size. An efficient flow control algorithm must control the input arrival process upstream the network (i.e. at the input access node). The leaky bucket technique is one example 22. In this paper we present and analyse a flow control algorithm based upon feedback throttling of the arrival process to the input statistical multiplexer. We apply a feedback control signal, proportional to the congestion level of the multiplexer, to the input source coder (see Figure 1). The signal will control the source rate by decreasing the coding rate (number of bits/sample). When another congestion threshold is achieved, a similar action is taken, thus reducing the input rate further. This algorithm is not of the reactive type, since it is applied at the input access node to the network, and its speed is not limited by the propagation delay. Any control action taken will be in time to alleviate the potential congestion. In fact we prevent congestion, since the feedback signal is proportional to the number of cells waiting in the multiplexer buffer. The control signal is transmitted using the ATM out-of-band signalling, hence it can be applied to sources directly connected to the network, as well as those connected through high-speed tANs. There are several advantages to this scheme. First, unlike the leaky bucket scheme it does not allow excess cells to enter the network. It prevents congestion, and greatly reduces the potential of congestion downstream in the network, since it reduces the number of cells per buffer. Second, it is applicable regardless of the type of coder used: it is good for variable bit rate coders as well as fixed bit rate ones. Third, it provides the means for the maximum possible shaping of the input arrival process through decreasing the peak bit rate. Consequently, the bandwidth allocated to the input call can be reduced, and yet the same required COS would be achieved. Also, the statistical multiplexing gain is enhanced, since more sources can be supported for each multiplexer, hence the traffic gets smoother and more easy to control. The scheme can thus be used to provide users with economically priced connections, since there are more users for each connection. The price to be paid is that we may perceive a slight degradation in the quality of the voice or video delivered, but we can prove that this degradation is quite marginal.
327
broadband ACCESS FLOW CONTROL IN VIDEO MULTLPLEXERS Modelling and analysis Figure 2 presents the continuous time Markov chain model of the video source, also known as the phase process. Maglarias et al. 23 presented and analysed this model to match the statistics of the continuous state autoregressive model. The autoregressive model is quite accurate in modelling the video source statistics. However, it is quite complicated if we try to use it in analytical studies. A similar model was used by Huang 24 to analyse the video statistics. Saito et aL 2s used a (J-state) Markov modulated process to study the performance of the video multiplexer. The model represents the arrival rate A(t) by quantizing the bit rate into uniform discrete levels, and the rate variations over time are approximated by a continuous time process with discrete jumps at random Poisson times. Thus the state space (A) of the chain represents a quantization level of the original sampled process, measured in bits/pixel. The (M + 1) states scan the range of the variations. The parameters a and /3 are the transitional rates of jumping from one quantization level to the other. These parameters were evaluated by Maglarias eta/. 23 by fitting them to the average, variance and the autocovariance functions of the original measured data of the source statistics. The results are: A = -CR(0) + -E(XD -
E(ZR) = 3.9/
(la)
M (lb)
1 +
a =3.9-~
(Ic)
where E(Z R) and CR(O) are the average and the variance of the aggregate arrival process from R identical and independent sources. Each source transmits a random process with mean E(A) and autocovariance function C ( t ' ) = C(O)e-3"9r. t" is the source frame number ndivided by a frame rate of 30 frames/s. The autocovariance curve was proved by several authors to follow an exponential fit. The value (3.9) was found to match the variations of this specific video experiment. The number of states M was set to be 10 R. It was found by Maglarias et al. 23 that this value of M yielded reasonable results that were close enough to the measured data. In this paper, all the variables used were normalized to cells/ms. The multiplexer buffer has a fixed buffer length of N
!
Figure 3.
!
i
i
Markov chain model
cells and is fed by the process described by the system of equations (I). Figure 3 shows the two dimension continuous time Markov chain model which describes the system's behaviour. The queue length stochastic process is a Markov process at instants of state changes. Each discrete level arrival to the queue is a Poisson process with exponential service time, with meanp where p =/_/C. C is the link capacity in bits/s, and L is the cell length in bits. The cell fixed service time is replaced by an exponential service time. Li 26 proved that replacing the service fixed time by an exponential time does not affect the queueing process, since the correlation effects of the arrival process dominate those of the service time process. Each state of the phase process iA, (0 < i < M), is therefore the equivalent of i sources, each having a Poisson arrival process at the rate of A cells/ms. The transitional rates between the system states are thus Poisson, of rate iA. Let KI and K2 be the queue lengths at which the flow control is activated. If the queue length reaches the threshold limit KI the rates drop to iB, and at queue length K2 the rates drop to iC. The rates B and C represent the arrival rates after decreasing the number of bits/sample of the source coder. In our analysis, we set those values to be 0.75A and 0.5A respectively. Let the duple {Q iA~}, where (,4 E~A, B, C) denote the number of cells in the queue and the phase of the arrival process, respectively. Then the stochastic equilibrium probability of the system is:
P×,v=Pr. {Q =x, ifi,=y} where (0 < i < M).
for(O
iC < y < i A )
We can write the following equations for the system:
MaPo, o = PPl.o + /~Po,A
A
Figure 2.
328
2A
(M -- 1 ) A
MA
Single video source model (phase process,)
for(x=O,y=O)
(2a)
{(M - i)a + iA + i~}Po, iA = I.IPI,iA + (M - i + 1)aPo, (i - 1)A + (i + 1)/3Po,(i + 1)A for ( l < i < M - 1 )
(2b)
(M• + MA)Po, MA
(2c)
= PPI,MA
+
CZPo,(M- 1)A
computer communications
broadband Ma if
for (1 < x < K1): (3a)
( M a + IJ)P×, o = laPx + 1, o 4" flPx, a
I(M - i ) a + i~ + i A + PlPx, i4 = ( M - i + 1)aPx,(i_ 1)4 + laPx + 1, iB -I- (i + 1)/3Px,(i + 1)A + iAPx - 1, hA, for ( l < i < M -1) (3b) (m~ 4- ~I)Px, MA = + PPx + 1, MB
(4a)
I(M - i ) a + ifl + iB + PIPx, iB = ( M - i + 1)aP×,(i_ 1)B + (i + 1)flPx,(i+ l)B + i B P x _ l,iB, -1)
(4b)
(M/~ +/~)P×, MB = aPx, (M - 1)B + M B P × _ 1, MB (4C)
+ PPx + 1, MC
for(K2+l
( M e + ,u)P×, o = PP× + 1, o + flPx, c
[(M - i ) a + i~ + i C + #tPx, iC = ( M - i + 1)aP×,(i_ 1)c 1)/~Px,(i + 1)c + iCP× _ 1, i o - 1)
+ PPx + 1, ic + (i +
for ( l < i < M (aft Jr t_l)Px, MC
4
.
(~P lh MI~
II I
M
u
(tvl I )(z -/I MI~
p
(M
M# MA
]b 1
H,N) =
(3C)
(Ma +/~)P×. 0 = PPx + 1,0 +/3P×. B
+ , u P x + 1,iC
-(kf - / b &-z q fl
=
a P x , (M _ 1)A 4"- M A P × _ 1,MA
for(K1+1
for ( l < i < M
T(ll
(5b)
- a P x , (M - 1)C "{" M C P x - 1, Mc
(5c)
+ PP× + 1, Mc
S is a diagonal matrix where p is its diagonal element. A is also a diagonal matrix where hA is its diagonal element. The matrices T(KI), T(K2) are the same structure as T(1) with the appropriate scaled )is. Similarly, A ( K I ) , A(K2) are the same as A but with the appropriate scaled Zs at each corresponding threshold levels KI and K2. The above matrices can be of extremely large sizes, yielding numerical difficulties in solving them. For example, if there are I 0 video sources and the buffer size is of 100 cells, then the dimension of matrix Q is 10 000, which is impossible to solve using direct matrix manipulations. We used matrix-geometric techniques, introduced by Neuts 27, to solve the above system. The solution uses an iteration refinement technique which needs to be slightly modified to suit the overload control in our case. The details are not repeated here, and the reader is referred to Neuts 27. Finally, the above equations were solved numerically for Px.v- The blocking probability PB is calculated from: M
for (x = N): (6a)
( M a + P)PN, 0 = flPN, C
PB = ~
PN, iC
(7)
i=0
[(m - i ) a + ifl + i C + P l P × , i c = ( m - i + lbP×,(i- 1)c + PPx + l , i c + (i + 1)~P×,(i+ l ) c + i C P × _ l , i O
for ( l < i < M
-1)
(6b)
( M ~ + P)Px, MC = aPx, (M - 1)C + M C P x - 1, Mc
+ PPx + I, MC
(6C)
The above equations can be written in the matrix form PQ -- 0, where P is the steady state probabilities vector and Q is the transitional rates matrix; then we have: 7((I)
H1)
A 5
Q=
T(KT)
A
5
T(K1)
A(K1) 5
T(K2) S
A(KT) T(K2)
A(K2) 5
T(M
T(0), T(1), T(KT), T(K2), T ( N ) , S, A, A(KT), A(K2) are all square matrices of the dimension M, where:
vol 15 no 5 june 1992
NUMERICAL RESULTS The video source characteristics reported above had an average arrival rate of 3.9 Mbits/s and a peak rate of 11 Mbits/s. We used a buffer length of 20 cells to limit the delay to 50 ps, where the cell length is the ATM standard of 53 bytes and C is 150 Mbits/s. Figure 4 reflects the improvement in the multiplexer performance for a different number of sources. The blocking probability has dropped significantly as a result of applying the flow control technique. As the buffer size is relatively small, the statistical multiplexing gain is not very appreciable. Figures 5 and 6 illustrate this effect more clearly. The trend is clear; as the buffer size is increased, the statistical multiplexing gain becomes more effective. However, as expected, the flow control technique has significantly enhanced the multiplexing gain. Figures 7 and 8 compare the performance of the multiplexer for different flow control thresholds. As the threshold levels are decreased, the performance improves; however, the price will be a slight degradation of image quality. The results reported here confirm our earlier discussions. It proves the imnportance of applying this type of flow control technique to accommodate sources with high peak rates without sacrificing the
329
broadband 0.001
0.0~5 0.01
.*
.'"
/ ,...
~ -.•
o O.
0.001
." • •
.90 190
•-
..
..-'
/
.- .....
/
..-
/
.-"
/ /
/"
,s-
.f'"
•.-
.i
.
0.0~3
.• .
.
.
""'i,
/"
.i./f./
9
/"
/
0.0002
-4.
./"
0.0001 •"
..
/
•
.'"
/
,i"
IE-05
1 E-06
f f /I
/
/-
/../ /./ ./
"'--
.. ,. -....
/.,/
%..
0.0001 -
.,,-
~
I
I
I
I
0.5
0.6
0.7
0.8
0.9
1.0
I 3
5E-05 2
Offered load
Figure4. B/ocking probability versus /oad (buffer size = 20). : no control, '/source; ---: no control, 2 sources; • • • : no control, 4 sources; - - - : w i t h c o n t r o l , 1 source; . . . . . : with control, 2 sources; -----: with
4
Number of sources
Figure 6. Statistical multiplexing gain - with control : buffer size = 10; ---: buffer (utilization = 0.8). s i z e = 15; • • • : buffer s i z e = 2 0
control, 4 sources
0.1 0.03 0.025
0.001 ° ° * ,....-
0.02
°°
.-"
••
:~
0.015
o
1E-05
. . . .iiiiiiiiiiiii ...
•
2Q.
.**
1E-07
O.Ol
.-
t f
/
,..
/
•
/
.
/
..'//// /
O_
0
•
• [
..."
14"
.
/
r':':
133
...
. .... ....,... ..... ....,.
1 E-09
it':
ss: ¢..
1E-11 0.005
Statistical
(utilization = 0.8).
3
multiplexing gain
-
: buffer size = 10; s i z e = '/5; • • • : b u f f e r s i z e = 2 0
no
---:
control buffer
efficiency. In our work we have placed a stringent delay limit of 50 ps. However, we can relax this value to I 0 0 Ms without having a major effect on the delay requirement. Therefore, we can double the buffer size and the statistical multiplexing gain with be more effective• The obvious result of decreasing the blocking probability will have the direct impact of minimizing serious congestion problems• Thus, we can efficiently utilize the valuable
330
..........
I
2
Number of sources
F i g u r e 5.
~-!
1E-13 L . 0.2
0•4
0•6
0.8
1
Offered load
Figure 7. Effect of flow contro/ thresho/d /eve/s, single source (buffer size = 20). : no control; - - - : K1 = 8, K 2 = 14; • • • : K 1 = 6 , K 2 = ' / 2 ; - - - : K 1 =4,/(2=8
network resources such as bandwidth and, in the meantime, provide different users with the required performance• Another impact is that we can accommodate more sources at the same bandwidth when flow control is not used. It had been suggested 11 that for bursty traffic with a high peak to link radio, the non-statistical
computer communications
broadband 0.1 ~-~7~.
0.0001
/
1E-07
/ /
.o
O
/ /
.
/ /,,. /..." /..' , /.,: ]..." / "
1E-10 I 1:"
, ,
1E-13
1E-16 I 0.2
0.4
0.6
q 0.8
Offered load Figure 8. Effect of flow control threshold levels, 4 sources (buffer size = 20). : no control; - - - : K1 = 4, K2 = 8; • • . : K 1 = 6 , K 2 = 1 2 ; - - - : K 1 = 8 , K2 = 14
operation mode could be more effective• We believe that with our proposed flow control algorithm we can utilize the VBR coding techniques to operate within the statistical multiplexing region with high efficiency•
the high frequency components due to sudden or abrupt motion in the image, which cannot be dropped. We have found that this flow control algorithm has several important merits in solving the problems of congestion in integrated networks. Besides preventing congestion, it has a major impact on the problem of bandwidth assignment and enforcement. We have addressed the problem of congestion control, and tried to devise unified hierarchical architecture that reflects the dependance and efforts of various network parameters on the performance• Traffic characteristics, call acceptance decision, bandwidth assignment, link traffic utilization and dynamic routing are some of the parameters that influence the design of an efficient flow control algorithm. We have tried to shed some light to this fact. The problem of flow control in integrated networks needs more work to achieve a novel scheme, if there is any. Our work calls for further studies, and some of the interesting outstanding issues are: 1 Performance analysis study of the bandwidth assignment rule with feedback flow control. 2 Problem of optimizing the statistical multiplexing gain with flow control under buffer size constraints. 3 Merits of applying this flow control algorithm at integrated switching nodes with heterogeneous input traffic, and its effects on switch performance• 4 Study of other traffic shaping techniques to decrease the burstiness and smooth out traffic characteristics.
REFERENCES CONCLUSIONS
AND FUTURE WORK
In this paper we have presented and analysed a flow control algorithm applied to the input access node of an integrated network• It operates on the principle of feedback control• The algorithm has been applied to the video cell multiplexer, where the arrival rate is controlled by reducing the number of bits per sample• When the first congestion level is reached, the rate is decreased from 8 to 6 bits/sample, and when the second congestion level is reached, the rate is decreased to 4 bits/sample. The algorithm throttles the peak arrival rate via a simple technique which does not depend upon particular detailed time dependant characteristics of the arrival process• Because the scheme is applied at the input access node, it is not limited by the propagation delay which is dominant in high-speed networks. Very little control can be done along the transit nodes, so it is essential to smooth down the burstiness of the input traffic to avoid the congestion caused by the formation of long bursts inside the network. The most effective method to decrease the burstiness, and hence avoid congestion, is by throttling the peak arrival rate. Cell dropping is not favourable, since it is difficult to distinguish these cells carrying the high resolution image information, which can be dropped, from those carrying
vol 15 no 5 june 1992
1 Coudreuse, J P, Thomas, A and Servel M, 'ATD techniques: An experimental packet network integrating videocommunication', ISS (1984) 2 Turner, J S and Wyatt, L F 'A packet network architecture for integrated services', Proc. GLOBECOMM (1983) 3 Reiner, H Draft Recommendation I. 12"/, CCITT.SGXVIII.TD•no 49, CCITT, Geneva, Switzerland (1988) 4 De Prycker, M, Plehiers, P, Fastrez, M and Bauwens, J 'Evolution towards a Belgian broadband experiment', (1987) 5 De Prycker, M 'Definition of network options for Belgian ATM broadband experiment', IEEEJ. Selected Areas in Commun., (December 1988) 6 Wernik, M 'Architecture and technology considerations for multimedia broadband communications', Proc. GLOBECOMM (1988) 7 Rider, M 'Protocols for ATM access networks', Proc. GLOBECOMM (1988) 8 Nomura, M, Fujii, T and Ohta, N 'Basic characteristics of variable rate video coding in ATM environment', IEEE J. Selected Areas in Commun., (June 1989) 9 Yasuda, Y, Yasuda, H, Ohla, N and Kishing, F 'Packet video transmission through ATM networks, Proc. GLOBECOMM (1989)
331
broadband 10 Verbiest, W, Pinnoo, L and Voeten, B 'Statistical multiplexing of variable bit rate video sources in ATM networks', Proc. GLOBECOMM (1988) 11 Woodruff, G, Rogers, R and Richards, P'Congestion control framework for high speed integrated packetized transport', Proc. GLOBECOIVlM (1988) 12 Takahashi, T and Hiramalsu, A 'Integrated ATM traffic control by neural networks, ISS (1990) 13 Goleslani, S 'Congestion free transmission of real time traffic in packet networks', Proc. INFOCOMM (1990) 14 Cidon, I, Sohraby, K and Bala, K'Congestion control for high speed packet switched networks', Proc. INFOCOMM (1990) 15 Eckberg, A E, Lucantoni, D and Luan, D'Meetingthe challenge: Congestion and flow control strategies for broadband information transport', Proc. GLOBECOMM (1989) 16 Decina, M, Toniatti, T, Vaccari, P and Verri, L 'Bandwidth assignment and virtual call blocking in ATM networks', Proc. INFOCOMM (1990) 17 Gallasi, G, Rigolio, G and Fratta, L 'ATM: Bandwidth assigment and enforcement policies', Proc. GLOBECOMM (1989) 18 Decina, M and Tonialti, T 'On bandwidth allocation to virtual bursty connections in ATM networks', Proc. /CC (1990) 19 Hui, J Y 'Resource allocation for broadband networks', IEEE J. Selected Areas in Commun., (December, 1988) 20 Ohnishi, H, Okada, T and Noguchi, K 'Flow control schemes and delay-loss tradeoff in ATM networks', IEEE J. Selected Areas in Cornmun., (December 1988)
332
21 Wang, W, Saadawi, T and Aihara, K 'Bandwidth variation and control for ATM networks', Proc. ICC (1990)
22 Sidi, M, Liu, M, Cidon, I and Gopal, I 'Congestion control
through
input
rate
regulation',
Proc.
GLOBECOMM (1989)
23 Maglarias, B, Anastassiou, D, Sen, P, Karlsson, G and Robbins, J 'Performance models of statistical multiplexing in packet video communications' IEEE Trans. Commun., (July 1988) 24 Huang, S 'Modeling and analysis for packet video', Proc. GLOBECOMM (1989) 25 Saito, H, Kawarazaki, M and Yamada, H 'Analysis of statistical multiplexing in ATM transport networks', Proc. ICC (1990) 26 Li, S O 'Overload control in a finite message buffer', IEEE Trans. Commun., (December 1989) 27 Neuts, M F Matrix-geometric solutions in stochastic models: An algorithmic approach, John Hopkins Press, New York (1981) 28 Goodman, D J 'Embedded DPCM forvariable bit rate transmission', IEEE Trans. Commun., (July 1980) 29 Brady, P T 'A Statistical analysis of on-off patterns in 16 conversation', Bell SysL Techn. ]., (January 1968) pp 73-91 30 Sriram, K and Whilt, W 'Characterizing superposition arrival processes in packet multiplexers for voice and data', IEEE J. Selected Areas in Commun., (September 1986) 31 Sriram, K and Lucantoni, D M 'Traffic smoothing effects of bit dropping in a packet voice multiplexer', IEEE Trans. Commun., (July 1989) 32 Gross, D and Harris, C M Fundamentals of Queueing Theory, John Wiley, New York (1985)
computer communications