- Email: [email protected]
Long range dependence and heavy tail distributions
Performance Evaluation 61 (2005) 91–93
Editorial
Long range dependence and heavy tail distributions One of the most important research topics in performance modeling and evaluation in the last decade has been the study of long range dependency (LRD) and heavy tail distributions of the Internet and Web traffic. Since the milestone publication of Leland et al. in 1993 (SIGCOMM 1993), significant research effort has been spent on traffic characterization and modeling using long range dependent processes and heavy tailed distributions. In parallel, queueing analyses with such forms of input traffic have received a great deal of attention by researchers of the performance evaluation community. This special issue brings together nine research papers providing a comprehensive review of these issues and their impact in the Internet, Web services and applications. Eight of these papers were selected from the 18 papers submitted to the special issue. All the submitted papers received at least three reviews each. The paper of Xia et al. was initially submitted to the Performance Evaluation journal as a regular paper. After it was accepted for publication, the Editor-in-Chief recommended its inclusion in this special issue. These nine papers span a wide range of topics in this hot research area. Their main contributions are outlined below. The relationship between Transmission Control Protocol (TCP) and long range dependence has been the topic of many research papers in recent years. Kherani and Kumar consider an Internet link carrying http-like traffic, i.e., transfers of finite volume files arriving at random time instants. These file transfers are controlled by an Adaptive Window Protocol (AWP), as in TCP. They provide analysis for the autocovariance function of the AWP controlled traffic into the link’s buffer. The analysis establishes that, for TCP controlled transfer of Pareto distributed file sizes with infinite second moment, the traffic into the link buffer is long range dependent. They also develop an analysis for obtaining the stationary distribution of the link buffer occupancy under an AWP controlled transfer of files sampled from some distribution. The analysis also provides a necessary and a sufficient condition for the existence of the finite mean link buffer content. These conditions have explicit dependence on the AWP used and the file size distribution, implying the sensitivity of the buffer occupancy process to the file size distribution. The work presented in this paper indicates, in particular, that the buffer behavior in the Internet may not be as poor as predicted from an open loop analysis of a queue fed with LRD traffic. Figueiredo, Liu, Feldmann, Misra, Towsley and Willinger also considers the relationship of TCP and self-similar traffic. They re-examine the same TCP trace that was used by Veres and Boda (“The chaotic nature of TCP congestion control”, in Proceedings of INFOCOM 2000) to claim that TCP creates selfsimilar traffic. A careful reassessment of their data analysis shows that this claim is not justified and
0166-5316/$ – see front matter © 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.peva.2005.04.001
92
Editorial / Performance Evaluation 61 (2005) 91–93
suggests that the TCP trace in question is not consistent with (asymptotic second-order) self-similarity or long range dependence. Figueiredo et al. also show that the traffic generated by a long-lived TCP connection, while exhibiting pronounced correlations over a predictable finite range of time scales, cannot be (asymptotically second-order) self-similar or exhibit LRD. This work serves as a reminder of the importance of careful trace analysis and detailed examination (and cross-validation) of alternative explanations when establishing or characterizing the generality of any particular finding about Internet traffic. Bekker, Borst and N´un˜ ez-Queija consider a fixed number of streaming sessions which share a bottleneck link with a dynamic population of elastic flows. Motivated by extensive measurement studies, they assume that the sizes of the elastic flows exhibit heavy tailed characteristics. The elastic flows are TCP controlled, while the transmission rates of the streaming applications are governed by a so-called TCP-friendly rate control protocol. Using the Processor-Sharing (PS) discipline to model the bandwidth sharing, Bekker et al. investigate the tail distribution of the deficit in service received by the streaming sessions compared to a nominal service target, and provide qualitative insight into the occurrence of persistent quality disruption for the streaming users. Traffic analysis continues to receive significant attention in this research field with the goals to better characterize workload and to better understand its impact. Yuan and Mills consider spatial–temporal characteristics of traffic in large-scale networks, and propose a method that reduces the amount of data needed while simultaneously retaining the ability to monitor spatial–temporal behavior network-wide. Motivated by insights about network dynamics at the macroscopic level, Mills and Yuan define a weight vector to build up information about the influence of local behavior over the whole network. By taking advantage of increased correlations arising in large networks, this method might require only a few observation points to capture shifting network-wide patterns over time. The performance of Web sites continues to be an important research topic. Such studies are invariably based on the access logs from the servers comprising the Web site. A problem with existing access logs is the coarse granularity of the timestamps, e.g., arrival times. Xia, Liu, Squillante, Zhang and Malouch consider Web traffic modeling at finer time scales and performance implication. They demonstrate and quantify the significant differences in performance obtained under diverse assumptions about the arrival process of user requests derived from the access logs, where the corresponding user response times can differ by more than an order of magnitude. This motivates the need for a general methodology to construct accurate representations of the actual arrival process of user requests from existing coarse-grained accesslog data. Xia et al. propose a drill-down methodology for constructing the arrival process at finer time scales based on the self-similar properties of the arrival process observed at coarse logging time scales. The advantage of this approach is that it maintains consistency between the properties of the arrival processes at both coarser and finer time scales. The remaining part of this special issue is devoted to the queueing analysis. Kim, Nam and Sung consider a single server fluid queueing system with fractional Brownian input, and show that there is a scaling property among the stationary queue-length distributions of different input parameters and service rates. As a consequence of the scaling property, they derive formulae for the effective bandwidth to guarantee the loss probability, and they analyze the shape of the admissible region and multiplexing gain. Mandjes and van Uitert consider the Generalized Processor-Sharing (GPS) mechanism serving two traffic classes. These classes consist of a large number of independent and identically distributed Gaussian flows with stationary increments. They apply Schilder’s sample-path large deviations theorem to calculate the logarithmic asymptotics of the upper and lower bounds. They then provide conditions under which
Editorial / Performance Evaluation 61 (2005) 91–93
93
the upper and lower bound match, and show how these results can be used to choose the values of the GPS weights. Recent experimental studies have shown that α-stable self-similar stochastic processes can accurately characterize various types of aggregate network traffic. Using this traffic modeling approach, LopezGuerrero, Orozco-Barbosa and Makrakis propose some probabilistic envelope processes that can be used to represent the resource demand of a traffic stream, and they design admission control mechanism for data and video traffic. Their analysis show that the presence of heavy tails in the distribution of a traffic process has a severe impact on the dimensioning of network elements. Borst, van Ooteghem and Zwart derive the sojourn time asymptotics for a multi-class GI/GI/1 queue with regularly varying service requirements operating under the Discriminatory Processor-Sharing (DPS) discipline. DPS provides a natural approach for modeling the level performance of differentiated bandwidth-sharing mechanisms. Under certain assumptions, Borst et al. prove that the service requirement and sojourn time of a given class have similar tail behavior, independent of the specific values of the DPS weights. As a by-product, they obtain an extension of the tail equivalence for ordinary PS queues to non-Poisson arrivals. The results suggest that DPS offers a potential instrument for effectuating preferential treatment to high-priority classes, without inflicting excessive delays on low-priority classes. Finally, as the Guest Editor, I would like to express my gratitude to the Editor-in-Chief, Dr. Werner Bux, for hosting this special issue in the Performance Evaluation journal and for his guidance throughout its publication process. I also would like to thank all the authors and all the reviewers for their contributions to this special issue. Zhen Liu IBM T.J. Watson Research Center, P.O. Box 704 Yorktown Heights, NY 10598, USA E-mail address: [email protected]
Editorial
Long range dependence and heavy tail distributions One of the most important research topics in performance modeling and evaluation in the last decade has been the study of long range dependency (LRD) and heavy tail distributions of the Internet and Web traffic. Since the milestone publication of Leland et al. in 1993 (SIGCOMM 1993), significant research effort has been spent on traffic characterization and modeling using long range dependent processes and heavy tailed distributions. In parallel, queueing analyses with such forms of input traffic have received a great deal of attention by researchers of the performance evaluation community. This special issue brings together nine research papers providing a comprehensive review of these issues and their impact in the Internet, Web services and applications. Eight of these papers were selected from the 18 papers submitted to the special issue. All the submitted papers received at least three reviews each. The paper of Xia et al. was initially submitted to the Performance Evaluation journal as a regular paper. After it was accepted for publication, the Editor-in-Chief recommended its inclusion in this special issue. These nine papers span a wide range of topics in this hot research area. Their main contributions are outlined below. The relationship between Transmission Control Protocol (TCP) and long range dependence has been the topic of many research papers in recent years. Kherani and Kumar consider an Internet link carrying http-like traffic, i.e., transfers of finite volume files arriving at random time instants. These file transfers are controlled by an Adaptive Window Protocol (AWP), as in TCP. They provide analysis for the autocovariance function of the AWP controlled traffic into the link’s buffer. The analysis establishes that, for TCP controlled transfer of Pareto distributed file sizes with infinite second moment, the traffic into the link buffer is long range dependent. They also develop an analysis for obtaining the stationary distribution of the link buffer occupancy under an AWP controlled transfer of files sampled from some distribution. The analysis also provides a necessary and a sufficient condition for the existence of the finite mean link buffer content. These conditions have explicit dependence on the AWP used and the file size distribution, implying the sensitivity of the buffer occupancy process to the file size distribution. The work presented in this paper indicates, in particular, that the buffer behavior in the Internet may not be as poor as predicted from an open loop analysis of a queue fed with LRD traffic. Figueiredo, Liu, Feldmann, Misra, Towsley and Willinger also considers the relationship of TCP and self-similar traffic. They re-examine the same TCP trace that was used by Veres and Boda (“The chaotic nature of TCP congestion control”, in Proceedings of INFOCOM 2000) to claim that TCP creates selfsimilar traffic. A careful reassessment of their data analysis shows that this claim is not justified and
0166-5316/$ – see front matter © 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.peva.2005.04.001
92
Editorial / Performance Evaluation 61 (2005) 91–93
suggests that the TCP trace in question is not consistent with (asymptotic second-order) self-similarity or long range dependence. Figueiredo et al. also show that the traffic generated by a long-lived TCP connection, while exhibiting pronounced correlations over a predictable finite range of time scales, cannot be (asymptotically second-order) self-similar or exhibit LRD. This work serves as a reminder of the importance of careful trace analysis and detailed examination (and cross-validation) of alternative explanations when establishing or characterizing the generality of any particular finding about Internet traffic. Bekker, Borst and N´un˜ ez-Queija consider a fixed number of streaming sessions which share a bottleneck link with a dynamic population of elastic flows. Motivated by extensive measurement studies, they assume that the sizes of the elastic flows exhibit heavy tailed characteristics. The elastic flows are TCP controlled, while the transmission rates of the streaming applications are governed by a so-called TCP-friendly rate control protocol. Using the Processor-Sharing (PS) discipline to model the bandwidth sharing, Bekker et al. investigate the tail distribution of the deficit in service received by the streaming sessions compared to a nominal service target, and provide qualitative insight into the occurrence of persistent quality disruption for the streaming users. Traffic analysis continues to receive significant attention in this research field with the goals to better characterize workload and to better understand its impact. Yuan and Mills consider spatial–temporal characteristics of traffic in large-scale networks, and propose a method that reduces the amount of data needed while simultaneously retaining the ability to monitor spatial–temporal behavior network-wide. Motivated by insights about network dynamics at the macroscopic level, Mills and Yuan define a weight vector to build up information about the influence of local behavior over the whole network. By taking advantage of increased correlations arising in large networks, this method might require only a few observation points to capture shifting network-wide patterns over time. The performance of Web sites continues to be an important research topic. Such studies are invariably based on the access logs from the servers comprising the Web site. A problem with existing access logs is the coarse granularity of the timestamps, e.g., arrival times. Xia, Liu, Squillante, Zhang and Malouch consider Web traffic modeling at finer time scales and performance implication. They demonstrate and quantify the significant differences in performance obtained under diverse assumptions about the arrival process of user requests derived from the access logs, where the corresponding user response times can differ by more than an order of magnitude. This motivates the need for a general methodology to construct accurate representations of the actual arrival process of user requests from existing coarse-grained accesslog data. Xia et al. propose a drill-down methodology for constructing the arrival process at finer time scales based on the self-similar properties of the arrival process observed at coarse logging time scales. The advantage of this approach is that it maintains consistency between the properties of the arrival processes at both coarser and finer time scales. The remaining part of this special issue is devoted to the queueing analysis. Kim, Nam and Sung consider a single server fluid queueing system with fractional Brownian input, and show that there is a scaling property among the stationary queue-length distributions of different input parameters and service rates. As a consequence of the scaling property, they derive formulae for the effective bandwidth to guarantee the loss probability, and they analyze the shape of the admissible region and multiplexing gain. Mandjes and van Uitert consider the Generalized Processor-Sharing (GPS) mechanism serving two traffic classes. These classes consist of a large number of independent and identically distributed Gaussian flows with stationary increments. They apply Schilder’s sample-path large deviations theorem to calculate the logarithmic asymptotics of the upper and lower bounds. They then provide conditions under which
Editorial / Performance Evaluation 61 (2005) 91–93
93
the upper and lower bound match, and show how these results can be used to choose the values of the GPS weights. Recent experimental studies have shown that α-stable self-similar stochastic processes can accurately characterize various types of aggregate network traffic. Using this traffic modeling approach, LopezGuerrero, Orozco-Barbosa and Makrakis propose some probabilistic envelope processes that can be used to represent the resource demand of a traffic stream, and they design admission control mechanism for data and video traffic. Their analysis show that the presence of heavy tails in the distribution of a traffic process has a severe impact on the dimensioning of network elements. Borst, van Ooteghem and Zwart derive the sojourn time asymptotics for a multi-class GI/GI/1 queue with regularly varying service requirements operating under the Discriminatory Processor-Sharing (DPS) discipline. DPS provides a natural approach for modeling the level performance of differentiated bandwidth-sharing mechanisms. Under certain assumptions, Borst et al. prove that the service requirement and sojourn time of a given class have similar tail behavior, independent of the specific values of the DPS weights. As a by-product, they obtain an extension of the tail equivalence for ordinary PS queues to non-Poisson arrivals. The results suggest that DPS offers a potential instrument for effectuating preferential treatment to high-priority classes, without inflicting excessive delays on low-priority classes. Finally, as the Guest Editor, I would like to express my gratitude to the Editor-in-Chief, Dr. Werner Bux, for hosting this special issue in the Performance Evaluation journal and for his guidance throughout its publication process. I also would like to thank all the authors and all the reviewers for their contributions to this special issue. Zhen Liu IBM T.J. Watson Research Center, P.O. Box 704 Yorktown Heights, NY 10598, USA E-mail address: [email protected]