Minstrel PIE: Curtailing queue delay in unresponsive traffic environments

Accepted Manuscript Minstrel PIE: Curtailing queue delay in unresponsive traffic environments Sachin D. Patil, Mohit P. Tahiliani

PII: DOI: Reference:

S0140-3664(18)30352-9 https://doi.org/10.1016/j.comcom.2019.03.006 COMCOM 5835

To appear in:

Computer Communications

Received date : 11 April 2018 Revised date : 26 December 2018 Accepted date : 10 March 2019 Please cite this article as: S.D. Patil and M.P. Tahiliani, Minstrel PIE: Curtailing queue delay in unresponsive traffic environments, Computer Communications (2019), https://doi.org/10.1016/j.comcom.2019.03.006 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Minstrel PIE: Curtailing Queue Delay in Unresponsive Traffic Environments Sachin D. Patil, Mohit P. Tahiliani Wireless Information Networking Group (WiNG), Department of Computer Science and Engineering, NITK Surathkal, Mangalore, Karnataka 575025, India

Abstract Active Queue Management (AQM) algorithms aim to maintain a proper tradeoff between queue delay and bottleneck link utilization. However, it is often noticed that this trade-off is not achieved convincingly when unresponsive UDP flows coexist with responsive TCP flows. This paper proposes an extension to Proportional Integral controller Enhanced (PIE) algorithm called Minstrel PIE, which adapts the reference queue delay to improve the trade-off between queue delay and link utilization when unresponsive flows share the same bottleneck queue as responsive flows. Extensive evaluations through simulations and real time experiments demonstrate that Minstrel PIE improves the performance of PIE in the presence of unresponsive flows, and delivers similar performance otherwise. Moreover, the Minstrel PIE algorithm does not introduce new knobs to improve the performance of PIE and hence, can be easily deployed without any additional complexity. Keywords: AQM algorithms, PIE, reference queue delay

1. Introduction There has been a proliferation of high capacity routers in the Internet to appropriately utilize the high speed data links. These routers typically consist Email addresses: [email protected] (Sachin D. Patil), [email protected] (Mohit P. Tahiliani)

Preprint submitted to Computer Communications

March 15, 2019

of very large buffers because the memory prices have fallen sharply. Due to 5

surplus buffering, packets experience excessive queuing delay which leads to significant degradation in the overall performance and largely reduces the quality of service for time critical applications. The above mentioned problem is known as the Bufferbloat problem [1]. Active Queue Management (AQM) is a promising technique to minimize the impact of Bufferbloat and improve the quality of

10

service for time critical applications [2]. Several AQM algorithms have been designed to monitor and limit the growth of the queue at routers. Random Early Detection (RED) [3], Random Exponential Marking (REM) [4], Adaptive RED (ARED) [5], BLUE [6], Proportional Integral controller (PI) [7], Controlled Delay (CoDel) [8], Proportional Integral controller Enhanced (PIE) [9], etc are

15

some of the well studied AQM algorithms. One of the vital characteristics of AQM algorithms is to maintain a proper trade-off between queue delay and bottleneck link utilization. But maintaining this trade-off is challenging when some of the flows crossing the router do not respond to the congestion feedback provided by AQM mechanisms, e.g., con-

20

gestion agnostic UDP flows do not respond to congestion notifications. The following are the major concerns in such cases: • Tackling the problem of Bufferbloat becomes challenging because unresponsive flows affect the stability of the queue. • The unresponsive behavior of congestion agnostic UDP flows defeats the

25

purpose of deploying AQM algorithms and leads to excessive queuing delays. • The problem further aggravates when unresponsive flows arrive in bursts, leading to variations in the Round Trip Time (RTT). The burst may also lead to bulk packet losses due to queue overflow.

30

• The network resource allocation is unfair because TCP flows respond to congestion signals whereas unresponsive flows do not. Hence, the major

2

share of the bandwidth gets utilized by unresponsive flows, leaving negligible room for TCP flows. Addressing these concerns is becoming increasingly important due to the 35

widespread adoption of UDP as the primary transport protocol. Popular applications of UDP include: DNS, routing updates, SNMP and NAT traversal techniques such as STUN (RFC 5389) and TURN (RFC 5766). Emerging applications of UDP include: video streaming, online gaming, peer to peer applications, Quick UDP Internet Connection (QUIC) [10] and browser frameworks

40

such as Web Real-Time Communication (WebRTC)1 . Although the emerging applications of UDP are obliged to implement congestion control at the application level and respond to congestion (e.g., QUIC and WebRTC), a significant share of the UDP traffic comprises of short-lived unresponsive flows. For example, a recent analysis of 4G LTE traffic from a

45

Tier-1 wireless carrier in the south-central US reveals that DNS dominates the UDP traffic in terms of the number of flows (39% of the total flows captured in the network) [12]. Despite the fact that DNS traffic is thin in terms of volume and might not lead to long lasting congestion, studies have shown that a such small fraction of unresponsive UDP flows can affect the queue stability and dete-

50

riorate the performance of AQM algorithms [13–15]. Our preliminary work [16] on analyzing the performance of ARED, CoDel and PIE also confirms that controlling queue occupancy is a non-trivial task when responsive and unresponsive flows coexist. In this paper, we propose a simple and easy-to-deploy extension to the PIE

55

algorithm, called Minstrel PIE, to increase its robustness when unresponsive flows coexist with TCP flows. We decided to base our work on PIE algorithm because: (i) it is a defacto AQM used by CableLabs since DOCSIS 3.1 cable modems (RFC 8034), and (ii) studies have shown that PIE is likely to provide 1 WebRTC

is a framework of protocols and JavaScript APIs that enables peer-to-peer audio,

video, and data sharing between browsers. It is supported by popular browsers such as Google Chrome and Mozilla Firefox, and it uses UDP for data transfers [11].

3

better control on queue delay than other algorithms in case of extreme overload 60

[17, 18]. Minstrel PIE adapts itself and timely drops (or marks)2 packets to control the queue delay when the traffic load increases, otherwise operates similar to PIE. We regulate the reference queue delay parameter of PIE to adjust the drop probability in Minstrel PIE. Depending on the results obtained through simulations and real time tests we note that Minstrel PIE maintains a better trade-off

65

between queue delay and link utilization when unresponsive flows coexist with TCP flows. It performs similar to PIE in other network settings. Typically, AQM algorithms are expected to avoid congestion and minimize the impact of Bufferbloat, whereas, protecting the fair share of responsive flows when they coexist with unresponsive flows is the responsibility of packet schedul-

70

ing algorithms (e.g., Stochastic Fair Queuing (SFQ) [19], Deficit Round Robin (DRR) [20]). Minstrel PIE, being an AQM algorithm, is mainly designed to avoid network congestion and minimize the impact of Bufferbloat. Recently, there has been an active interest in designing hybrid packet scheduler and AQM algorithms to collectively address the problems of congestion, Bufferbloat and

75

flow protection by giving each flow its own queue. Flow Queue CoDel (FQCoDel) (RFC 8290) and Flow Queue PIE (FQ-PIE) [21] are two attempts in this direction. Along the similar lines, we combine Minstrel PIE with flow queuing to provide flow protection and name the resulting algorithm as Flow Queue Minstrel PIE (FQ-MinstrelPIE). Besides thoroughly evaluating the work-

80

ing of Minstrel PIE, this paper additionally provides a brief evaluation of FQMinstrelPIE. The paper is organized as follows: Section 2 presents the background of PIE algorithm and discusses the impact of using fixed reference queue delay. Section 3 presents the design and implementation of Minstrel PIE and FQ-MinstrelPIE.

85

Section 4 discusses the results obtained from simulations and real time tests and 2 Marking

refers to the setting of a bit in the packet header by using congestion signaling

approaches like Explicit Congestion Notification (ECN) (RFC 3168). The words ‘drops’ and ‘marks’ are used interchangeably in this paper.

4

Section 5 concludes the paper with future directions.

2. Background 2.1. Overview of PIE The functionality of PIE is based on the original Proportional Integral (PI) 90

controller [22] that attempts to control the queue occupancy around a desired reference queue length. Unlike PI, PIE aims to control the queue delay around a desired reference queue delay. PIE obtains an estimate of the current queue delay and compares it with the reference queue delay and previous queue delay to calculate the packet drop

95

probability (p) as shown in the following equation: p(i) = p(i−1) + α ∗ (cur delay − qdelay ref ) + β ∗ (cur delay − old delay) (1) where, • p(i) indicates the drop probability for the current interval. • p(i−1) indicates the drop probability for the previous interval. • cur delay is the current queue delay.

100

• qdelay ref is the reference queue delay. • old delay is the queue delay used in the previous sample. • α and β are auto-tuning scaling factors as described in RFC 8033. PIE uses Little’s Law [23] to obtain an estimate of the cur delay, as shown below: cur delay =

105

queue length avg dq rate

where, • queue length indicates the current queue occupancy. • avg dq rate is the average dequeue rate of the queue. 5

(2)

avg dq rate is an exponentially weighted moving average which provides a fairly good estimate of the bottleneck link utilization. We use this estimate in 110

Minstrel PIE to adapt qdelay ref. The main goal of PIE is to maintain the cur delay around a pre-fixed qdelay ref. It uses auto-tuning scaling factors α and β to make necessary adjustments to p and minimize the difference between cur delay and qdelay ref. When cur delay becomes equal to qdelay ref and old delay, PIE reaches a steady state

115

and continues to drop the packets with a fixed p. p is calculated at a regular interval of t update3 . 2.2. Impact of fixed qdelay ref Despite gaining significant attention, the implications of keeping qdelay ref fixed in PIE are not widely studied. In this section, we highlight the need to adapt qdelay ref in PIE. We configure a dumbbell topology as shown in Fig. 1 10 Mbps, 40 ms

100 Mbps, 5 ms

100 Mbps, 5 ms

Sender 1

Receiver 1

Sender 2

Receiver 2

Sender 3

PIE / Minstrel PIE

Droptail

Sender n

Receiver 3

Receiver n

Figure 1: Dumbbell Topology used in ns-2 experiments 120

in ns-2 [24] with different traffic loads to analyze the impact of having a fixed value of qdelay ref. Three traffic loads are configured in the simulation: (i) Light traffic (5 TCP flows), (ii) Heavy traffic (50 TCP flows) and (iii) Mix traffic (5 TCP and 2 UDP flows). These configurations are identical to the ones used 125

by the authors of PIE [9] and PI2 [25], except that we have used Compound TCP [26, 27] for our simulations instead of TCP NewReno (RFC 6582). PIE 3 16ms

as mentioned in RFC 8033.

6

has been configured with the values recommended in RFC 8033. In this paper, all the simulations are run for 100 seconds and the results presented in Fig. 2 through Fig. 12 depict the average values obtained by repeating the simulation with 25 different random seed values.

1

1

0.9

0.8 0.7

0.6

CDF

CDF

0.7 0.5

0.6 0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

1

2

3

4

5

Light TCP Heavy TCP Mix TCP and UDP Reference Delay

0.9

Light TCP Heavy TCP Mix TCP and UDP

0.8

6

7

8

9

0

10

0

5

10 15 20 25 30 35 40 45 50

Link Utilization (%)

Queue Delay (ms)

(a) Link Utilization

(b) Queue Delay

Figure 2: Link Utilization and Queuing Delay with PIE

1

1

0.9

0.9

Light TCP Heavy TCP Mix TCP and UDP

0.8 0.7

0.7

0.6 0.5

0.6 0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

1

2

3

4

5

Light TCP Heavy TCP Mix TCP and UDP Reference Delay

0.8

CDF

CDF

130

6

7

8

9

0

10

0

10

20

30

40

50

Queue Delay (ms)

Link Utilization (%)

(a) Link Utilization

(b) Queue Delay

Figure 3: Link Utilization and Queuing Delay with Minstrel PIE

Fig. 2 presents the CDF for link utilization and queue delay obtained with PIE. Increasing the number of responsive flows did not largely impact the performance, but adding a few unresponsive flows led to a significant rise in the queue delay. This happens because PIE does not drop sufficient number of pack7

135

ets to keep the queue delay under control. We believe that this concern can be addressed by adapting qdelay ref (albeit within acceptable bounds) and subsequently regulating the packet drop probability (p) to drop appropriate number of packets. This approach forms the basis of Minstrel PIE. We repeat the simulations described above with Minstrel PIE. Fig. 3 presents

140

the CDF for link utilization and queue delay. As traffic load increases, Minstrel PIE increases its packet drop probability based on qdelay ref adjustments and achieves better control on queue delay without hurting the link utilization. In the next section, we describe the working of Minstrel PIE.

3. Minstrel PIE 145

3.1. Design The design of Minstrel PIE is tailored to improve the tradeoff between queue delay and link utilization when responsive and unresponsive flows coexist. It increases qdelay ref when the link is underutilized and decreases qdelay ref when the link is over-utilized avg dq rate measurements in PIE serve as a good ap-

150

proximation of the current link utilization. Minstrel PIE leverages these measurements and accordingly adapts the qdelay ref. The initial value of qdelay ref is set to 15ms which is a fixed default value suggested in RFC 8033. Other components in Minstrel PIE are same as PIE. Algorithm 1 presents the pseudo code of the proposed algorithm.

155

A new parameter called maximum average dequeue rate (maxavg dq rate) is introduced in Minstrel PIE to keep track of the maximum link utilization observed so far. At every t update interval, avg dq rate is compared with maxavg dq rate to decide whether qdelay ref should be updated. Subsequently, the updated value of qdelay ref is used to calculate the new drop probability.

160

maxavg dq rate is internally maintained by Minstrel PIE and does not require settings from the user. The following subsections provide a detailed insight into the working of Minstrel PIE:

8

Algorithm 1: Minstrel PIE Input : avg dq rate Initialization: qdelay ref = 15ms, t update = 16ms, maxavg dq rate = 0 Output 1

: updated value of qdelay ref

On every t update interval // Track the highest avg dq rate

2 3

if avg dq rate > maxavg dq rate then maxavg dq rate = avg dq rate

4

end

5

if avg dq rate <= 0.9 ∗ maxavg dq rate then

6 7

// Increase qdelay ref |qdelay ref − cur delay| qdelay ref + = 2 end

8

else // Decrease qdelay ref

9

if cur delay < qdelay ref then qdelay ref = cur delay

10 11

end

12

else qdelay ref − =

13 14

|qdelay ref − cur delay| 2

end

15

end

16

if qdelay ref < 5ms then

17

qdelay ref = 5ms

18

end

19

else if qdelay ref > 15ms then

20

qdelay ref = 15ms

21

end

22

Calculate Drop Probability p 9

3.1.1. Increase qdelay ref 165

Minstrel PIE keeps track of maxavg dq rate and increases the qdelay ref when the link is not adequately utilized. Specifically, it increases the qdelay ref when the avg dq rate ≤ 90% of the maxavg dq rate. A binary increment (as shown in Algorithm 1) is applied to the value of qdelay ref. This increment helps Minstrel PIE to adjust the difference between cur delay and qdelay ref which

170

is tracked by PIE’s control parameter α during the drop probability calculation. These adjustments ensure that the overall drop probability is decreased quickly, and subsequently, the link utilization increases. The upper bound for qdelay ref is set to 15ms in Minstrel PIE. The design considerations for choosing this setting and the above mentioned increase/decrease threshold (i.e., 90% of

175

maxavg dq rate) are discussed later in Section 3.2. 3.1.2. Decrease qdelay ref Minstrel PIE decreases the qdelay ref when the link is adequately utilized. Specifically, it decreases the qdelay ref when avg dq rate > 90% of the maxavg dq rate. However, decreasing the value of qdelay ref abruptly may hurt the

180

link utilization; so the following approach has been adopted to decrease the value of qdelay ref smoothly: If cur delay < qdelay ref, decreasing qdelay ref and setting it to cur delay is safe. But if cur delay > qdelay ref, a binary decrement is applied to decrease the value of qdelay ref smoothly. In the former case, Minstrel PIE attempts

185

to stabilize the value of drop probability (p) by setting qdelay ref to cur delay whereas in the latter case, Minstrel PIE attempts to gradually increase the drop probability (p) by increasing the value of (cur delay - qdelay ref ). The motivation to use binary decrement is to ensure quick adaptation of qdelay ref to achieve the desired trade-off. The lower bound for qdelay ref is set to 5ms in

190

Minstrel PIE. Section 3.2 discusses the design considerations for choosing this value.

10

3.2. Parameter Settings 3.2.1. Lower and Upper Bound for qdelay ref The inferences to set lower and upper bound for qdelay ref in Minstrel PIE 195

have been derived from the simulation studies presented in [18] where the qdelay ref in PIE is set within a range of 5ms to 20ms. Hence, the lower bound for qdelay ref in Minstrel PIE is set to 5ms, which is also the default value used by CoDel. The upper bound for qdelay ref in Minstrel is set to 15ms instead of 20ms. The primary reason is that higher values of qdelay ref exhibit more

200

variations in the queue delay (See Figure 7(b) of [18]). Moreover, RFC 8033 recommends a default value of 15ms for qdelay ref. Thus, Minstrel PIE does not deviate from the primary goals of PIE. 3.2.2. Initial value of qdelay ref Initially, the buffers are empty and it takes a few RTTs for the queue to

205

build up. Setting a low initial value for qdelay ref would be too restrictive and it might not allow the link utilization to grow beyond a certain limit. Hence, we recommend that Minstrel PIE sets the initial value of qdelay ref to its upper bound, i.e., 15ms. 3.2.3. Setting the increase/decrease threshold

210

Quite a few prior works have demonstrated that PIE achieves more than 90% link utilization in most of the scenarios [18, 28, 29]. Based on the observations made in the literature, and our own evaluations, we recommend 90% of maxavg dq rate to be used as a threshold for Minstrel PIE to increase / decrease the value of qdelay ref.

215

3.3. Support for Explicit Congestion Notification and Flow Queuing Explicit Congestion Notification (ECN) (RFC 3168) mechanism marks packet headers (TCP and IP) to notify endpoints of congestion that may be developing in a bottleneck queue, without resorting to packet drops. Minstrel PIE does not require modifications to support ECN style packet marking. It is recommended

11

220

to use ECN marking with Minstrel PIE to avoid any loss of throughput for TCP flows, especially when qdelay ref is decreased from 15ms to 5ms. The term flow queuing (RFC 8290) implies that every flow has its own queue so that the time sensitive flows are isolated from those that tend to fill the queue or affect the queue stability. Minstrel PIE, like CoDel and PIE, can be combined

225

with flow queuing to provide flow protection. Although there are several ways, in this paper, we follow the approach adopted in FQ-CoDel and FQ-PIE to combine flow queuing with Minstrel PIE. A hashing based scheme is used to provide a separate queue to every flow, and hashing is performed based on the following five tuples: source and destination IP addresses, source and destination port

230

numbers and protocol number. A modified DRR as described in RFC 8290 is used for queue scheduling and Minstrel PIE algorithm operates on every queue. Combining flow queuing with Minstrel PIE offers further benefits by punishing the flows that tend to affect the queue stability and deteriorate the performance of other competing flows.

235

3.4. Implementation To verify the correctness of the proposed idea, PIE code in ns-2.36.rc1, ns3.27 and Linux kernel 4.15 has been modified to support Minstrel PIE. It requires 18, 37 and 33 lines of code change to PIE implementation in ns-2 [24], ns-3 [30] and Linux kernel, respectively. Moreover, Minstrel PIE does not add any new

240

parameter which requires explicit settings from the user. In the Linux kernel, the net/sched/sch pie.c is modified to implement Minstrel PIE and to enable Minstrel PIE through the tc command, q pie.c is modified in the iproute2 package. To enable Minstrel PIE, passing ‘minstrel’ as a parameter in the tc-pie command is sufficient4 .

245

Implementing FQ-MinstrelPIE was not straightforward because, unlike FQCoDel, the Linux kernel does not contain an in-built module for FQ-PIE. Al4 Example:

sudo tc qdisc add dev eno1 parent 1:10 handle 1100: pie limit 200 target 15ms

tupdate 16ms minstrel

12

though a preliminary implementation of FQ-PIE is available publicly [31], it has not been validated and tested. Hence, we implemented a new module for FQ-PIE in the Linux kernel and it has been made available publicly at [32]. Sub250

sequently, we implemented FQ-MinstrelPIE in the Linux kernel for our work.

4. Evaluation Our evaluations consisted of three sets of experiments that compare the performance of Minstrel PIE with PIE using (i) preliminary set of simulation scenarios, same as the ones used in [8][9][25] (ii) simulation scenarios described 255

in RFC 7928 for comparing the performance of AQM algorithms and (iii) real time tests provided in Flexible Network Tester (Flent) [33] for evaluating AQM algorithms. Besides analyzing the trade-off between latency and link utilization, we compare the fairness achieved with PIE and Minstrel PIE to show that the latter does not deteriorate the fairness among the flows. Jains Fairness Index

260

[34] has been used to calculate the fairness. 4.1. Preliminary Evaluation The preliminary evaluation consisted of two types of scenarios: (i) simulations with different number of responsive and unresponsive flows, and (ii) simulations in realistic network conditions, with a proper mix of responsive and

265

unresponsive flows. The scenarios mentioned in Section 4.1.1 to Section 4.1.3 are same as the ones described in Section 2.2, except that we represent the results in two additional forms: distribution of bottleneck link utilization and queue delay over time, and goodput as a function of queue delay which is recommended in RFC 7928. ns-2.36.rc1 has been used for these simulations.

270

4.1.1. Light TCP traffic This scenario consists of 5 TCP flows starting at the same time in a dumbbell topology as shown in Fig. 1. Table 1 presents the configuration of simulation parameters for this scenario. Fig. 4 shows the performance of PIE and Minstrel

13

Table 1: Simulation Configuration for Preliminary Evaluation Parameters

Values

Bottleneck Propagation Delay

Values

80 ms

Traffic Flows Direction

Forward Only

Non-bottleneck Propagation Delay

20 ms

AQM enabled in Forward path

PIE/Minstrel PIE

Bottleneck Bandwidth

10 Mbps

TCP Application

Long-lived FTP

Non Bottleneck Bandwidth

100 Mbps

UDP Application with rate

Constant Bit Rate at 10Mbps

Buffer Size

200 packets

TCP enabled

Compound

Packet Size

1000 Bytes

Simulation Time

100 Seconds

180

180

PIE qdelay_ref

MinstrelPIE qdelay_ref (Upper limit) qdelay_ref (Lower limit)

160 Queuing Delay (in ms)

160 Queuing Delay (in ms)

Parameters

140 120 100 80 60 40 20

140 120 100 80 60 40 20

0

0 0

10

20

30 40 50 60 70 Time (in seconds)

80

90 100

0

10

(a) PIE

20

30 40 50 60 70 Time (in seconds)

80

90 100

(b) Minstrel PIE

Figure 4: Queuing Delay for Light TCP traffic

PIE in terms of queue delay and Fig. 5 shows the performance of PIE and 275

Minstrel PIE in terms of bottleneck link utilization. The sharp rise in queue delay in Fig. 4 and the sudden fall in the link utilization in Fig. 5 is because all flows start at the same time. This configuration provides an insight into the capabilities of PIE and Minstrel PIE to handle the burst of traffic. Both algorithms handle the burst appropriately and maintain

280

the queue delay around the reference values for the rest of the simulation. Minstrel PIE has marginally better control over the queue delay with same link utilization (Fig. 4(a) and Fig. 4(b)) as that of PIE. This is because Minstrel PIE tries to reduce the queue delay to the lower limit (seen around 15 seconds

14

10

9

9

Link Utilization (in Mbps)

Link Utilization (in Mbps)

10 8 7 6 5 4 3 2 1

8 7 6 5 4 3 2 1

0

0 0

10

20

30 40 50 60 70 Time (in seconds)

80

90 100

0

10

20

(a) PIE

30 40 50 60 70 Time (in seconds)

80

90 100

(b) Minstrel PIE

Figure 5: Link Utilization for Light TCP traffic Table 2: Fairness for Light TCP traffic scenario

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

TCP 2

TCP 3

TCP 4

TCP 5

PIE

1.84

1.83

1.87

1.85

1.85

0.99

MinstrelPIE

1.83

1.84

1.80

1.80

1.83

0.99

in Fig. 4(b)) since the link is sufficiently utilized, and allows the queue delay 285

to grow towards the upper limit (seen around 65 seconds in Fig. 4(b)) when it senses the link utilization getting affected (15 seconds to 65 seconds in Fig. 5(b)). Furthermore, the results shown in Table 2 confirm that Minstrel PIE does not affect the fairness among the flows. 4.1.2. Heavy TCP traffic

290

This scenario sets up 50 TCP flows in a dumbbell topology, but otherwise, it is same as the previous scenario. Fig. 6(a) and Fig. 6(b) show the performance of PIE and Minstrel PIE in terms of queue delay, respectively and Fig. 7(a) and Fig. 7(b) show the performance of PIE and Minstrel PIE in terms of bottleneck link utilization.

295

Like Fig. 4, there is a sharp rise in queue delay in Fig. 6. Even though the amount of burst in this scenario is much larger than the previous one, it is observed that both algorithms successfully control the queue delay. After the burst

15

180

MinstrelPIE qdelay_ref (Upper limit) qdelay_ref (Lower limit)

160 Queuing Delay (in ms)

Queuing Delay (in ms)

180

PIE qdelay_ref

160 140 120 100 80 60 40 20

140 120 100 80 60 40 20

0

0 0

10

20

30 40 50 60 70 Time (in seconds)

80

90 100

0

10

(a) PIE

20

30 40 50 60 70 Time (in seconds)

80

90 100

80

90 100

(b) Minstrel PIE

10

10

9

9

Link Utilization (in Mbps)

Link Utilization (in Mbps)

Figure 6: Queuing Delay for Heavy TCP traffic

8 7 6 5 4 3 2 1

8 7 6 5 4 3 2 1

0

0 0

10

20

30 40 50 60 70 Time (in seconds)

80

90 100

(a) PIE

0

10

20

30 40 50 60 70 Time (in seconds)

(b) Minstrel PIE

Figure 7: Link Utilization for Heavy TCP traffic

period, PIE maintains the queue delay around the qdelay ref whereas Minstrel PIE maintains the queue delay between the lower and upper limit. Since there 300

are sufficient number of TCP flows to keep the link fully utilized, Minstrel PIE tries to maintain the queue delay near to its lower bound of qdelay ref. In terms of link utilization and fairness among the flows (See Table 3), both algorithms have a similar performance. 4.1.3. Mix TCP and UDP traffic

305

This is an extension of the Light TCP traffic scenario, where 2 UDP flows are added to 5 TCP flows. The goal is to verify the ability of Minstrel PIE 16

180

MinstrelPIE qdelay_ref (Upper limit) qdelay_ref (Lower limit)

160 Queuing Delay (in ms)

Queuing Delay (in ms)

180

PIE qdelay_ref

160 140 120 100 80 60 40 20

140 120 100 80 60 40 20

0

0 0

10

20

30 40 50 60 70 Time (in seconds)

80

90 100

0

10

(a) PIE

20

30 40 50 60 70 Time (in seconds)

80

90 100

80

90 100

(b) Minstrel PIE

10

10

9

9

Link Utilization (in Mbps)

Link Utilization (in Mbps)

Figure 8: Queuing Delay for Mix TCP and UDP traffic

8 7 6 5 4 3 2 1

8 7 6 5 4 3 2 1

0

0 0

10

20

30 40 50 60 70 Time (in seconds)

80

90 100

(a) PIE

0

10

20

30 40 50 60 70 Time (in seconds)

(b) Minstrel PIE

Figure 9: Link Utilization for Mix TCP and UDP traffic

17

Table 3: Fairness for Heavy TCP traffic scenario

Flow Number

Throughput (in Mbps) TCP 1-10

TCP 11-20

TCP 21-30

TCP 31-40

TCP 41-50

0.19

0.19

0.19

0.19

0.18

0.19

0.18

0.18

0.18

0.19

0.19

0.19

0.19

0.19

0.19

0.18

0.18

0.19

0.19

0.19

0.18

0.19

0.19

0.19

0.19

0.19

0.18

0.19

0.19

0.19

0.18

0.19

0.18

0.19

0.19

0.19

0.19

0.19

0.19

0.19

0.18

0.19

0.19

0.19

0.19

0.19

0.19

0.19

0.19

0.19

0.18

0.19

0.17

0.19

0.18

0.19

0.18

0.19

0.18

0.19

0.17

0.19

0.18

0.19

0.18

0.19

0.19

0.19

0.19

0.19

Minstrel

0.18

0.19

0.19

0.19

0.19

PIE

0.18

0.18

0.19

0.18

0.19

0.18

0.19

0.18

0.19

0.19

0.19

0.18

0.19

0.18

0.19

0.18

0.19

0.18

0.19

0.18

0.19

0.19

0.18

0.19

0.19

PIE

Fairness

0.98

0.99

algorithm to control queue delay in the presence of unresponsive flows. Fig. 8(a) and Fig. 8(b) show the performance of PIE and Minstrel PIE in terms of queue delay, respectively and Fig. 9(a) and Fig. 9(b) show the performance of PIE and Minstrel PIE in terms of bottleneck link utilization. Table 4: Fairness for Mix TCP and UDP traffic scenario

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

TCP 2

TCP 3

TCP 4

TCP 5

UDP 1

UDP 2

PIE

0.010

0.015

0.006

0.007

0.014

4.790

4.729

0.288

Minstrel PIE

0.010

0.015

0.006

0.007

0.020

4.788

4.729

0.289

310

Although the initial burst of traffic is successfully handled by both algorithms, PIE is unable to maintain the queue delay within its qdelay ref for the rest of the simulation. On the other hand, Minstrel PIE consistently maintains the queue delay around its lower bound of qdelay ref without hurting the link

18

315

utilization. Since the link utilization consistently remains at its peak (100%), Minstrel PIE senses an opportunity to minimize the queue delay as much as possible. A low value in the ‘Fairness’ column in Table 4 for both algorithms highlights the unfairness problem due to the unresponsive nature of UDP flows. This

320

problem can be addressed by combining flow queuing, without compromising on the benefits offered by Minstrel PIE. Section 4.3.3 provides more details.

11

PIE MinstrelPIE

10.5

Goodput (Mbps)

Goodput (Mbps)

11

10 9.5 9 8.5

PIE MinstrelPIE

10.5 10 9.5 9 8.5

8

8 25

20

15 10 5 0 Queue Delay (ms)

-5

-10

25

(a) Light TCP

20

15 10 5 0 Queue Delay (ms)

-5

-10

(b) Heavy TCP

Goodput (Mbps)

11 10.5 10 9.5 9 8.5

PIE MinstrelPIE

8 25

20

15 10 5 0 Queue Delay (ms)

-5

-10

(c) Mix TCP and UDP Figure 10: Representation of Fig. 4 to Fig. 9 as per RFC 7928

RFC 7928 recommends plotting the goodput as a function of queue delay to compare the performance of AQM algorithms. Fig. 10 depicts the results of above three scenarios as per the RFC 7928 guidelines. An ellipse represents 325

the contour with maximum likelihood 2D Gaussian distribution of goodput and

19

delay. The height and width of the ellipse are a function of standard deviation of goodput and delay, respectively. The orientation of an ellipse is based on the covariance between the two. Since the center of an ellipse represents the mean with equal extension of height and width on both sides, it is possible to get part 330

of the contour in the region with negative queue delay or with goodput greater than bottleneck link capacity. The area inside an ellipse can be interpreted as the variance in the values of goodput and delay. The more the variations, the larger is the outline of an ellipse [35]. The results show that Minstrel PIE maintains a better tradeoff, especially when responsive and unresponsive traffic

335

coexist. 4.2. RFC 7928 based Evaluation Table 5: Simulation Configuration for RFC 7928 based Evaluation Parameters Bottleneck Propagation Delay Non-bottleneck Propagation Delay

Values 80ms 20ms

Parameters Traffic Flows Direction AQM enabled in Forward path

Values Forward Only PIE/Minstrel PIE

Bottleneck Bandwidth

10 Mbps

TCP Application

Long-lived FTP

Non Bottleneck Bandwidth

100 Mbps

UDP Application with rate

Constant Bit Rate at 10Mbps

Buffer Size

200 packets

TCP enabled

NewReno

Packet Size

1000 Bytes

Simulation Time

300 Seconds

Thorough evaluation of an AQM algorithm requires significant time and efforts. Hence, the AQM and Packet Scheduling Working Group at IETF published RFC 7928 which provides the guidelines to evaluate AQM algorithms. An 340

automated evaluation framework complying with the guidelines of RFC 7928 has been recently developed for ns-3 [36]. We use this evaluation suite to verify the desired behavior of Minstrel PIE and its relative performance against PIE under various traffic conditions. While RFC 7928 recommends several test scenarios to evaluate AQM algorithms, we present the results for the ones that are most

20

345

relevant to the objective of designing Minstrel PIE. Table 5 presents the simulation configuration used by the AQM Evaluation Suite of ns-3. The same has been used for the evaluation of Minstrel PIE. 4.2.1. Unresponsive transport sender Table 6: Scenarios mentioned in Section 5.3 of RFC 7928 Traffic Scenario

Consists of a long-lived UDP flow from an unresponsive

Mix TCP

UDP Transport Sender

application with a single long-lived, non-application limited

and UDP

TCP NewReno flow.

With single

Consists of a non-application limited and long-lived, single

UDP Sender

UDP flow with sending rate more than bottleneck capacity.

9.4

9.4

9.2

9.2

Goodput (Mbps)

Goodput (Mbps)

Description

9 8.8 8.6

PIE MinstrelPIE

8.4 16

14

12 10 8 Queue Delay (ms)

PIE MinstrelPIE

9 8.8 8.6 8.4

6

4

30

e

(a) UDP with TCP Mix

25

20 15 10 5 Queue Delay (ms)

0

-5

(b) Single UDP Sender

Figure 11: Results for Section 5.3 from RFC 7928

The scenarios mentioned in this section are identical to the ones mentioned 350

in Section 5.3 of RFC 7928. The main idea is to evaluate the performance of AQM algorithms in the presence of unresponsive transport. Table 6 presents a summary of the scenarios. Fig. 11 presents the results for the traffic scenarios mentioned in Table 6. Minstrel PIE shows a tight control on the queue delay in both the scenarios,

355

more so in the case of a single unresponsive flow. We observe that both algorithms provide a stable performance (i.e., variations in queue delay and link

21

Table 7: Fairness for Section 5.3 traffic scenario from RFC7928

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

UDP 1

PIE

0.063

8.983

0.50

Minstrel PIE

0.074

9.171

0.50

utilization are negligible in Fig. 11(b)) because a single UDP flow has been configured with Constant Bit Rate (CBR) application, which sends the data at a constant rate. The utilization and fairness (See Table 7) obtained with both 360

the algorithms is similar. These results are in line with those discussed in the previous section. 4.2.2. Levels of network congestion Table 8: Congestion scenarios mentioned in Section 8 of RFC 7928 Congestion Scenarios 1

Mild Congestion

2

Medium Congestion

3

Heavy Congestion

Description Consists of bulk non-application limited TCP flow which can generate packet drop rate of 0.1% at the router. Consists of bulk non-application limited TCP flow which can generate packet drop rate of 0.5% at the router. Consists of bulk non-application limited TCP flow which can generate packet drop rate of 1% at the router.

RFC 7928 specifies three scenarios with different levels of congestion varying from mild, medium to heavy congestion, respectively. The main idea is 365

to evaluate the performance of AQM algorithms in different levels of network congestion. Table 8 presents a summary of the scenarios. The details of each scenario configuration can be found in Section 8.2.2, 8.2.3 and 8.2.4 of RFC 7928, respectively. Since these scenarios configure only responsive flows to vary the amount of

370

congestion in the network, Minstrel PIE and PIE are expected to provide similar performance. Fig. 12(a), Fig. 12(b) and Fig. 12(c) depict the results for above mentioned scenarios, respectively.

22

9.2

9.4

PIE MinstrelPIE Goodput (Mbps)

Goodput (Mbps)

9.4

9 8.8 8.6 8.4

PIE MinstrelPIE

9.2 9 8.8 8.6 8.4

30 25 20 15 10 5 0 -5 -10 -15 Queue Delay (ms)

30 25 20 15 10 5 0 -5 -10 -15 Queue Delay (ms)

(a) Mild Congestion

Goodput (Mbps)

9.4

(b) Medium Congestion

PIE MinstrelPIE

9.2 9 8.8 8.6 8.4

30 25 20 15 10 5 0 -5 -10 -15 Queue Delay (ms)

(c) Heavy Congestion Figure 12: Results for Section 8.2.2 to 8.2.4 from RFC 7928

4.3. Evaluation using Flent To verify the effectiveness of Minstrel PIE in a real network environment, we 375

conducted experiments on a testbed by using FLExible Network Tester (Flent) [33]. Flent is developed to overcome the problems of coordination among different testing tools such as netperf [37] and iperf [38], reproducing testbed results, managing testbed configuration, and storing and analysing measurement data [33]. It provides test scenarios to evaluate the performance of AQM algorithms.

380

Flent’s TCP upload (tcp up) test has been used to evaluate the performance of Minstrel PIE. The testbed comprises of 1 sender, 1 receiver and 1 AQM machine, all running Ubuntu 16.04 LTS with Linux kernel 4.15. Minstrel PIE is implemented only on the AQM machine. Flent specific parameters settings and

23

traffic loads are listed in Table 9. In addition to evaluating Minstrel PIE in basic 385

scenarios like light and heavy TCP traffic, we evaluate it in realistic scenarios like: on-off unresponsive UDP flow, ECN enabled TCP flows, with and without flow queuing and different bottleneck buffer sizes. Table 9: Testbed Setup using ethtool, netem, tc and Flent Parameters

Values

Bottleneck Propagation Delay

Parameters

Values

Traffic Flows

80 ms

Forward Only

Direction

Non-bottleneck Propagation Delay

20 ms

AQM enabled in Forward path

PIE/Minstrel PIE

Bottleneck Bandwidth

10 Mbps

TCP Application

Long-lived FTP

Non Bottleneck Bandwidth

100 Mbps

UDP Application with rate

Constant Bit Rate at 10Mbps

Buffer Size

200 packets

TCP enabled

CUBIC (RFC 8312)

Packet Size

1000 Bytes

Simulation Time

100 Seconds

180 160 140 120 100 80 60 40 20 0

10

PIE MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

4.3.1. Light TCP traffic

8 6 4 2

PIE MinstrelPIE

0 0

20

40

60 Time (s)

80

100

0

(a) Queue Delay

20

40

60 Time (s)

80

100

(b) TCP Throughput

Figure 13: Light TCP Traffic

This scenario is identical to the one discussed in Section 4.1.1 and consists 390

of 5 TCP flows that start at the same time in a dumbbell topology. Fig. 13 (a) and Fig. 13 (b) compare the performance of PIE and Minstrel PIE in terms of queue delay and bottleneck link utilization, respectively. The slight rise initially 24

Table 10: Fairness for Light TCP traffic scenario in testbed

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

TCP 2

TCP 3

TCP 4

TCP 5

PIE

2.13

2.01

1.73

1.88

1.75

0.99

MinstrelPIE

1.94

1.94

1.93

1.84

1.83

0.99

in queue delay in Fig. 13 (a) is because all flows start at the same time. Both algorithms have similar performance since all the flows are responsive flows and 395

moreover, the number of flows is less. Additionally, Table 10 shows that the fairness among flows remains same with both the algorithms.

180 160 140 120 100 80 60 40 20 0

10

PIE MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

4.3.2. Heavy TCP traffic

8 6 4 2

PIE MinstrelPIE

0 0

20

40 60 Time (s)

80

100

0

(a) Queue Delay

20

40 60 Time (s)

80

100

(b) TCP Throughput

Figure 14: Heavy TCP Traffic

This scenario is identical to the one discussed in Section 4.1.2 and sets up 50 TCP flows in a dumbbell topology, but otherwise, it is same as the previous 400

scenario. Fig. 14 (a) and Fig. 14 (b) compare the performance of PIE and Minstrel PIE in terms of queue delay and bottleneck link utilization, respectively. Queue delay is initially higher than the previous scenario because the number of flows is increased to 50 and all start at the same time. Both algorithms control queue delay appropriately for the rest of the experiment. PIE maintains the

405

queue delay around its qdelay ref and Minstrel PIE maintains it between the lower and upper limits of qdelay ref. The bottleneck link remains fully utilized 25

Table 11: Fairness for Heavy TCP traffic scenario in testbed

Flow Number

PIE

MinstrelPIE

Throughput (in Mbps) TCP 1-10

TCP 11-20

TCP 21-30

TCP 31-40

TCP 41-50

0.18

0.18

0.21

0.17

0.21

0.18

0.21

0.18

0.17

0.16

0.18

0.20

0.18

0.18

0.19

0.19

0.21

0.20

0.18

0.22

0.19

0.19

0.19

0.18

0.18

0.19

0.21

0.18

0.19

0.20

0.20

0.18

0.22

0.18

0.21

0.18

0.16

0.20

0.19

0.19

0.18

0.20

0.20

0.17

0.21

0.19

0.21

0.19

0.20

0.21

0.21

0.21

0.18

0.2

0.19

0.16

0.17

0.22

0.18

0.18

0.18

0.19

0.21

0.20

0.21

0.19

0.18

0.20

0.19

0.20

0.21

0.17

0.21

0.21

0.19

0.19

0.16

0.17

0.19

0.19

0.19

0.18

0.20

0.19

0.19

0.22

0.19

0.17

0.19

0.19

0.20

0.18

0.18

0.20

0.20

0.19

0.18

0.20

0.17

0.19

Fairness

in Fig. 14 (b) due to a large number of TCP flows, and both algorithms have a similar performance. Furthermore, Table 11 confirms that Minstrel PIE does not deteriorate the fairness among flows. 410

4.3.3. Benefits of ECN and Flow Queuing This section highlights the benefits of using ECN and Flow Queuing with Minstrel PIE. The experiments consist of two traffic configurations: 1-TCP-1UDP (a single TCP flow with a single UDP flow), and 5-TCP-1-UDP (five TCP flows with a single UDP flow). TCP flows start at 1 second and run until the

415

end of the experiment. UDP flow starts at 25 seconds and stops at 75 seconds. These configurations help us to evaluate the performance of Minstrel PIE when unresponsive flows join and leave the network. Initially, we evaluate the benefits of using ECN with Minstrel PIE, followed by evaluating the benefits of using flow queuing. 26

0.99

0.99

10

PIE MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

180 160 140 120 100 80 60 40 20 0

PIE MinstrelPIE

8 6 4 2 0

0

20

40 60 Time (s)

80

100

0

(a) Queue Delay

20

40 60 Time (s)

80

100

(b) TCP Throughput

Figure 15: Mix TCP and UDP Traffic with tcp 1up Table 12: Fairness in tcp 1up test without ECN

Flow Number

420

Throughput (in Mbps)

Fairness

TCP 1

UDP 1

PIE

0.083

9.60

0.50

MinstrelPIE

0.064

9.61

0.50

Fig. 15 (a) and Fig. 15 (b) compare PIE and Minstrel PIE in terms of queue delay and TCP throughput without enabling ECN in a 1-TCP-1-UDP configuration, respectively. Large oscillation in queue delay around 25 seconds is due to the start of UDP flow. Minstrel PIE observes the increase in the avg dq rate and accordingly reduces the qdelay ref to achieve a tight control on

425

queue delay. Later, when UDP flow leaves the network at 75 seconds, Minstrel PIE increases the qdelay ref. Table 13: Fairness in tcp 1up test with ECN

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

UDP 1

PIE

0.13

9.54

0.51

MinstrelPIE

0.11

9.56

0.51

Fig. 15 (b) shows the impact on the throughput of TCP flow when UDP flow starts. TCP throughput reduces significantly when UDP flow starts, and gets back to normal when UDP flow stops. The results in Table 12 show the 27

10

PIE MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

180 160 140 120 100 80 60 40 20 0

PIE MinstrelPIE

8 6 4 2 0

0

20

40 60 Time (s)

80

100

0

(a) Queue Delay with ECN

20

40 60 Time (s)

80

100

(b) TCP Throughput with ECN

Figure 16: Mix TCP and UDP Traffic with tcp 1up

430

throughput share obtained by TCP and UDP flows. We repeat this experiment by enabling PIE and Minstrel PIE to use ECN, and the results are depicted in Fig. 16 (a) and Fig. 16 (b). It is observed that TCP throughput improves

180 160 140 120 100 80 60 40 20 0

10

PIE MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

marginally (See Table 13), but still gets largely affected when UDP flow is on.

PIE MinstrelPIE

8 6 4 2 0

0

20

40 60 Time (s)

80

100

0

(a) Queue Delay

20

40 60 Time (s)

80

100

(b) TCP Throughput

Figure 17: Mix TCP and UDP Traffic

Fig. 17 (a) and Fig. 17 (b) compare PIE and Minstrel PIE in terms of queue 435

delay and TCP throughput without enabling ECN in a 5-TCP-1-UDP configuration, respectively. The results are similar to those observed for 1-TCP-1-UDP configuration in Fig. 15. After a sharp reduction to zero, TCP throughput improves slightly with both the algorithms, but takes more time with Minstrel

28

Table 14: Fairness in tcp 5up test without ECN Throughput (in Mbps)

Fairness

TCP 1

TCP 2

TCP 3

TCP 4

TCP 5

UDP 1

PIE

0.19

0.16

0.18

0.18

0.22

8.50

0.24

MinstrelPIE

0.17

0.17

0.18

0.16

0.17

8.60

0.24

180 160 140 120 100 80 60 40 20 0

10

PIE MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

Flow Number

PIE MinstrelPIE

8 6 4 2 0

0

20

40 60 Time (s)

80

100

0

(a) Queue Delay with ECN

20

40 60 Time (s)

80

100

(b) TCP Throughput with ECN

Figure 18: Mix TCP and UDP Traffic with ECN

Table 15: Fairness in tcp 5up test with ECN

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

TCP 2

TCP 3

TCP 4

TCP 5

UDP 1

PIE

0.32

0.29

0.30

0.34

0.31

8.30

0.28

MinstrelPIE

0.31

0.30

0.31

0.32

0.32

8.32

0.28

29

PIE. The main reason for this delay is that the drop probability of Minstrel PIE 440

is higher than that of PIE on account of lower qdelay ref. However, this concern is resolved by marking TCP packets using ECN, instead of dropping, as shown in Fig. 18 (b). The slight improvement offered by ECN can be observed by

180 160 140 120 100 80 60 40 20 0

10

FQ-PIE FQ-MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

comparing the results shown in Table 14 and 15.

8 6 4 2

FQ-PIE FQ-MinstrelPIE

0 0

20

40 60 Time (s)

80

100

0

(a) Queue Delay

20

40 60 Time (s)

80

100

(b) TCP Throughput

Figure 19: Mix TCP and UDP Traffic with tcp 1up

Table 16: Fairness in tcp 1up test without ECN

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

UDP 1

FQ-PIE

4.70

4.80

0.99

FQ-MinstrelPIE

4.61

4.90

0.99

Both PIE and Minstrel PIE being AQM algorithms do not provide flow pro445

tection. As discussed earlier, combining these algorithms with flow queuing is a promising approach. We provide some preliminary insights into the benefits offered by combining Minstrel PIE with flow queuing and comparing its performance with FQ-PIE by using the same two configurations described above. Fig. 19 (a) shows that FQ-MinstrelPIE has lesser average queue delay when UDP

450

flow is on between 25 seconds to 75 seconds and Fig. 19 (b) shows that using flow queuing offers significant advantage in providing fair share to the TCP flow. Table 16 confirms that TCP flow achieves its fair share of throughput. Fig. 20 shows the results for the same experiment, but with ECN enabled. Fig. 30

10

FQ-PIE FQ-MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

180 160 140 120 100 80 60 40 20 0

8 6 4 2

FQ-PIE FQ-MinstrelPIE

0 0

20

40 60 Time (s)

80

100

0

(a) Queue Delay with ECN

20

40 60 Time (s)

80

100

(b) TCP Throughput with ECN

Figure 20: Mix TCP and UDP Traffic with tcp 1up Table 17: Fairness in tcp 1up test with ECN Throughput (in Mbps)

Flow Number

Fairness

TCP 1

UDP 1

FQ-PIE

4.58

4.9

0.99

FQ-MinstrelPIE

4.62

4.9

0.99

20 (a) shows that FQ-MinstrelPIE has lesser queue delay when UDP flow is on 455

and Fig. 20 (b) shows that enabling ECN helps to achieve a better stability in TCP throughput with FQ-MinstrelPIE. Table 17 shows the throughput share achieved by TCP and UDP flows. Table 18: Fairness in tcp 5up test without ECN

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

TCP 2

TCP 3

TCP 4

TCP 5

UDP 1

FQ-PIE

1.58

1.60

1.58

1.60

1.60

1.62

0.99

FQ-MinstrelPIE

1.60

1.59

1.60

1.58

1.57

1.64

0.99

To verify the working of flow queuing scheme, we consider the 5-TCP-1UDP configuration. Fig. 21(a) and 21(b) presents the average queue delay and 460

TCP throughput for FQ-PIE and FQ-MinstrelPIE without ECN, respectively. Since the number of TCP flows are more in this configuration, the aggregate TCP throughput achieved by PIE and Minstrel PIE is around 8 Mbps when UDP flow is on. Table 18 confirms that TCP flows get a fair share with flow 31

10

FQ-PIE FQ-MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

180 160 140 120 100 80 60 40 20 0

8 6 4 2

FQ-PIE FQ-MinstrelPIE

0 0

20

40 60 Time (s)

80

100

0

(a) Queue Delay

20

40 60 Time (s)

80

100

(b) TCP Throughput

Figure 21: Mix TCP and UDP Traffic with tcp 5up

queuing. Fig. 22 shows the results for the same experiment, but with ECN 465

enabled. Fig. 21 (a) and Fig. 22 (a) show that FQ-MinstrelPIE performs similar to FQ-PIE, and sometimes better when UDP flow is on. Fig. 21 (b) and Fig. 22 (b) show that both algorithms achieve similar TCP throughput, and Table 19 shows the throughput share achieved by TCP and UDP flows. Depending on these experimental observations we recommend to enable ECN and flow queuing, if possible, to achieve maximum advantage of using Minstrel PIE. The results reveal that Minstrel PIE is a promising algorithm, even when ECN and flow queuing are not enabled.

180 160 140 120 100 80 60 40 20 0

10

FQ-PIE FQ-MinstrelPIE

TCP Throughput (Mbps)

Queue Delay (ms)

470

8 6 4 2

FQ-PIE FQ-MinstrelPIE

0 0

20

40

60 Time (s)

80

100

0

(a) Queue Delay with ECN

20

40

60 Time (s)

80

(b) TCP Throughput with ECN

Figure 22: Mix TCP and UDP Traffic with tcp 5up

32

100

Table 19: Fairness in tcp 5up test with ECN

Flow Number

Throughput (in Mbps)

Fairness

TCP 1

TCP 2

TCP 3

TCP 4

TCP 5

UDP 1

FQ-PIE

1.58

1.60

1.57

1.57

1.58

1.62

0.99

FQ-MinstrelPIE

1.57

1.58

1.56

1.59

1.57

1.64

0.99

4.3.4. Evaluation with different buffer sizes

10

Upload

Latency (ms) MinstrelPIE-15 PIE-15 DropTail-15

Upload

10

Latency (ms) MinstrelPIE-100 PIE-100 DropTail-100

140

9

300

9 130

6

8 ms

120

ms

250

7

Mbits/s

Mbits/s

8

200 7

110

5

150 6

4 100

100

(a) Buffer 15 10

(b) Buffer 100 Upload

Latency (ms) MinstrelPIE-1000 PIE-1000 DropTail-1000

2000

8

ms

Mbits/s

1500 6

1000 4

500 2

(c) Buffer 1000 Figure 23: Light TCP Traffic with TCP CUBIC

A significant amount of work has been done to study the impact of bottleneck 475

buffer size on queue delay and link utilization [39–41]. In this scenario, we repeat the experiments described in Section 4.3.1 and 4.3.2 by configuring different buffer sizes at the bottleneck. We use the standard buffer size of 15, 100 and 1000 packets [42, 43] in both the cases. In addition, we also compare the results of PIE and Minstrel PIE with the traditional tail drop queues. Fig. 17 and Fig.

480

18 depict the results obtained with light TCP traffic and heavy TCP traffic, respectively.

33

10

9

Upload

Latency (ms) MinstrelPIE-15 PIE-15 DropTail-15

Upload

10

130

125

Latency (ms) MinstrelPIE-100 PIE-100 DropTail-100

8

250

8

6

6 200

ms

115

ms

7

Mbits/s

Mbits/s

120

4 110

5

150 105

4

2

100

(a) Buffer 15 10

(b) Buffer 100 Upload

Latency (ms) MinstrelPIE-1000 PIE-1000 DropTail-1000

2000

8 1500

1000

ms

Mbits/s

6

4

500

2

(c) Buffer 1000 Figure 24: Heavy TCP Traffic with TCP CUBIC

When the buffer size is 15 packets, it is observed that the RTT is marginally lesser with PIE and Minstrel PIE than tail drop queues. As the buffer size increases to 100 and 1000 packets, PIE and Minstrel PIE offer significant ad485

vantage over tail drop queues in terms of RTT. The link utilization is similar for all three algorithms when the buffer size is 15 packets and the traffic is light. However, the link utilization is marginally affected with tail drop queues when heavy traffic is used with a buffer size of 15 packets. This happens because the tail drop queues cause the large number

490

of TCP flows passing through this small buffer to synchronize. The impact of global synchronization on the overall link utilization is also visible in the cases where buffer size is 100, more so with 1000 packets because the congestion window of TCP flows grow to a large value. The link utilization with PIE and Minstrel PIE is better in all the cases. In general, Minstrel PIE provides a

495

better performance trade-off than PIE and tail drop queues.

34

5. Conclusions This paper proposed an extension to the PIE AQM algorithm, called Minstrel PIE. It adapts the qdelay ref to achieve a better trade-off between queue delay and link utilization. The results obtained from the simulation studies and 500

real time experiments validate the effectiveness and robustness of Minstrel PIE against unresponsive traffic. Minstrel PIE can be incrementally deployed in a real network setup as it is a minor modification of PIE, and does not require the user to configure any parameter explicitly. Furthermore, we demonstrated that FQ-MinstrelPIE ensures fair allocation of resources to responsive flows when

505

they coexist with unresponsive flows, and recommend that it is a suitable and effective algorithm to be investigated further for real time deployment.

References [1] J. Gettys, K. Nichols, Bufferbloat: Dark buffers in the Internet, Communications of the ACM 55 (1) (2012) 57–65. 510

[2] J. Gettys, Bufferbloat FAQ (2014). URL http://gettys.wordpress.com/bufferbloat-faq/ [3] S. Floyd, V. Jacobson, Random early detection gateways for congestion avoidance, IEEE/ACM Transactions on Networking 1 (4) (1993) 397–413. [4] S. Athuraliya, V. Li, S. Low, Q. Yin, REM: Active Queue Management,

515

in: Teletraffic Engineering in the Internet Era, Vol. 4 of Teletraffic Science and Engineering, Elsevier, 2001, pp. 817–828. [5] S. Floyd, R. Gummadi, S. Shenker, Adaptive RED: An algorithm for increasing the robustness of RED’s active queue management (2001). URL http://www.icir.org/floyd/papers/adaptiveRed.pdf

520

[6] W. Feng, K. G. Shin, D. D. Kandlur, D. Saha, The BLUE Active Queue Management Algorithms, IEEE/ACM Transactions on Networking (ToN) 10 (4) (2002) 513–528. 35

[7] C. V. Hollot, V. Misra, D. Towsley, W. Gong, On designing improved controllers for AQM routers supporting TCP flows, in: INFOCOM, Twentieth 525

Annual Joint Conference of the IEEE Computer and Communications Societies Proceedings, Vol. 3, IEEE, 2001, pp. 1726–1734. [8] K. Nichols, V. Jacobson, Controlling Queue Delay, Communications of the ACM 55 (7) (2012) 42–50. [9] R. Pan, P. Natarajan, C. Piglione, M. S. Prabhu, V. Subramanian,

530

F. Baker, B. VerSteeg, PIE: A lightweight control scheme to address the bufferbloat problem, in: IEEE 14th International Conference on High Performance Switching and Routing (HPSR), IEEE, 2013, pp. 148–155. [10] A. Langley, A. Riddoch, A. Wilk, A. Vicente, C. Krasic, D. Zhang, F. Yang, F. Kouranov, I. Swett, J. Iyengar, J. Bailey, J. Dorfman, J. Roskind, J. Ku-

535

lik, P. Westin, R. Tenneti, R. Shade, R. Hamilton, V. Vasiliev, W. Chang, Z. Shi, The QUIC Transport Protocol: Design and Internet-Scale Deployment, in: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, SIGCOMM ’17, ACM, New York, NY, USA, 2017, pp. 183–196.

540

[11] A. Bergkvist, D. C. Burnett, C. Jennings, A. Narayanan, B. Aboba, Webrtc 1.0: Real-time communication between browsers, Working draft, W3C 91, 2012. [12] F. Li, X. Jiang, J. W. Chung, M. Claypool, Who is the King of the Hill? Traffic Analysis over a 4G Network, in: IEEE International Conference on

545

Communications (ICC), 2018, pp. 1–6. [13] C. Hollot, Y. Liu, V. Misra, D. Towsley, Unresponsive flows and AQM performance, in: INFOCOM, Vol. 1, IEEE, 2003, pp. 85–95. [14] W. Li, L. Zeng-zhi, C. Yan-ping, X. Ke, Fluid-based stability analysis of mixed TCP and UDP traffic under RED, in: 10th IEEE International

36

550

Conference on Engineering of Complex Computer Systems, ICECCS Proceedings, IEEE, 2005, pp. 341–348. [15] J. Cai, Z. Zhang, X. Song, An analysis of UDP traffic classification, in: 12th IEEE International Conference on Communication Technology (ICCT), IEEE, 2010, pp. 116–119.

555

[16] S. D. Patil, M. P. Tahiliani, On the robustness of AQM mechanisms against non-responsive traffic, in: IEEE International Conference on Advanced Networks and Telecommunications Systems (ANTS), IEEE, 2016, pp. 1–6. doi:10.1109/ANTS.2016.7947857. [17] I. J¨arvinen, M. Kojo, Evaluating CoDel, PIE, and HRED AQM techniques

560

with load transients, in: IEEE 39th Conference on Local Computer Networks (LCN), IEEE, 2014, pp. 159–167. [18] N. Kuhn, D. Ros, A. B. Bagayoko, C. Kulatunga, G. Fairhurst, N. Khademi, Operating ranges, tunability and performance of CoDel and PIE, Computer Communications 103 (2017) 74–82.

565

[19] P. E. McKenney, Stochastic fairness queueing, in: Proceedings. IEEE INFOCOM ’90: Ninth Annual Joint Conference of the IEEE Computer and Communications Societies The Multiple Facets of Integration, Vol. 2, 1990, pp. 733–740. [20] M. Shreedhar, G. Varghese, Efficient fair queuing using Deficit Round-

570

Robin, IEEE/ACM Transactions on Networking 4 (3) (1996) 375–385. [21] R. Al-Saadi, G. Armitage, Dummynet AQM v0. 2–CoDel, FQ-CoDel, PIE and FQ-PIE for FreeBSD’s ipfw/dummynet framework, Centre for Advanced Internet Architectures, Swinburne University of Technology, Melbourne, Australia, Tech. Rep. A 160418 (2016) 18.

575

[22] H. Zhang, C. V. Hollot, D. Towsley, V. Misra, A self-tuning structure for adaptation in TCP/AQM networks, in: GLOBECOM ’03. IEEE Global

37

Telecommunications Conference (IEEE Cat. No.03CH37489), Vol. 7, 2003, pp. 3641–3646. doi:10.1109/GLOCOM.2003.1258913. [23] J. D. C. Little, S. C. Graves, Little’s Law, Springer US, Boston, MA, 2008, 580

pp. 81–100. doi:10.1007/978-0-387-73699-0 5. URL https://doi.org/10.1007/978-0-387-73699-0 5 [24] S. McCanne, S. Floyd, The LBNL network simulator (ns-2) (1997). URL http://www.isi.edu/nsnam [25] K. De Schepper, O. Bondarenko, I. Tsang, B. Briscoe, PI 2: A Linearized

585

AQM for both Classic and Scalable TCP, in: Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies, ACM, 2016, pp. 105–119. [26] K. Tan, J. Song, Compound TCP: A Scalable and TCP-friendly Congestion Control for High-speed Networks, in: 4th International workshop on

590

Protocols for Fast Long-Distance Networks (PFLDNet), 2006. [27] K. Tan, J. Song, Q. Zhang, M. Sridharan, A Compound TCP Approach for High-Speed and Long Distance Networks, in: 25th IEEE International Conference on Computer Communications, Proceedings IEEE INFOCOM, 2006, pp. 1–12.

595

[28] C. Kulatunga, N. Kuhn, G. Fairhurst, D. Ros, Tackling Bufferbloat in capacity-limited networks, in: European Conference on Networks and Communications (EuCNC), IEEE, 2015, pp. 381–385. [29] N. Kuhn, D. Ros, Improving PIE’s performance over high-delay paths, CoRR abs/1602.00569, 2016.

600

URL http://arxiv.org/abs/1602.00569 [30] T. R. Henderson, M. Lacage, G. F. Riley, C. Dowell, J. Kopena, Network simulations with the ns-3 simulator, SIGCOMM demonstration 14 (14) (2008) 527–527.

38

[31] H. Okano, FQ-PIE old implementation is available at (2017). 605

URL https://github.com/hironoriokano/fq-pie [32] R. Gautam, V.Saicharan, B. Mohit, M. Leslie, S. D. Patil, M. P. Tahiliani, FQ-PIE new implementation is available at (2018). URL https://github.com/gautamramk/FQ-PIE-for-Linux-Kernel [33] T. Høiland-Jørgensen, Flent: The FLExible Network Tester, in: The 11th

610

Swedish National Computer Networking Workshop (SNCNW), Karlstad, Sweden, May 28–29, 2015. [34] R. Jain, A. Durresi, G. Babic, Throughput fairness index: An explanation, (1999), Tech. rep., Department of CIS, The Ohio State University. [35] K. Winstein, H. Balakrishnan, TCP Ex Machina: Computer-generated

615

Congestion Control, in: Proceedings of the ACM, SIGCOMM ’13, ACM, New York, NY, USA, 2013, pp. 123–134. doi:10.1145/2486001.2486020. [36] A. Deepak, K. S. Shravya, M. P. Tahiliani, Design and Implementation of AQM Evaluation Suite for ns-3, in: Proceedings of the Workshop on ns-3, WNS3 ’17, ACM, New York, NY, USA, 2017, pp. 87–94.

620

[37] R. Jones, et al., NetPerf: a Network Performance Benchmark, Information Networks Division, Hewlett-Packard Company, (1996). URL https://hewlettpackard.github.io/netperf/ [38] A. Tirumala, F. Qin, J. Dugan, J. Ferguson, K. Gibbs, Iperf, (2006),. URL https://iperf.fr/

625

[39] D. Wischik, N. McKeown, Part I: Buffer sizes for core routers, Computer Communication Review 35 (2005) 75–78. [40] G. Raina, D. Towsley, D. Wischik, Part II: Control Theory for Buffer Sizing, SIGCOMM Comput. Commun. Rev. 35 (3) (2005) 79–82. doi:10.1145/ 1070873.1070885.

630

URL http://doi.acm.org/10.1145/1070873.1070885 39

[41] M. Enachescu, Y. Ganjali, A. Goel, N. McKeown, T. Roughgarden, Part III: Routers with Very Small Buffers, Computer Communication Review 35 (2005) 83–90. [42] G. Raina, D. Wischik, Buffer sizes for large multiplexers: TCP queueing 635

theory and instability analysis, in: Next Generation Internet Networks, 2005, pp. 173–180. doi:10.1109/NGI.2005.1431663. [43] G. Raina, S. Manjunath, S. Prasad, K. Giridhar, Stability and Performance Analysis of Compound TCP with REM and Drop-Tail Queue Management, IEEE/ACM Transactions on Networking 24 (4) (2016) 1961–1974. doi:

640

10.1109/TNET.2015.2448591.

40