CoCoA++: Delay gradient based congestion control for Internet of Things

Future Generation Computer Systems 100 (2019) 1053–1072

Contents lists available at ScienceDirect

Future Generation Computer Systems journal homepage: www.elsevier.com/locate/fgcs

CoCoA++: Delay gradient based congestion control for Internet of Things ∗

Vishal Rathod , Natasha Jeppu, Samanvita Sastry, Shruti Singala, Mohit P. Tahiliani Wireless Information Networking Group (WiNG), Department of Computer Science and Engineering, NITK Surathkal, Mangalore, Karnataka, 575025, India

highlights • • • •

A new congestion control algorithm for CoAP applications in Internet of Things. CAIA Delay Gradient (CDG) to predict network congestion. Probabilistic Backoff Factor (PBF) to control network congestion. Performance evaluation of CoCoA and CoCoA++ in static and mobile environments.

article

info

Article history: Received 10 April 2018 Received in revised form 31 March 2019 Accepted 26 April 2019 Available online 6 May 2019 Keywords: Congestion control CoAP Internet of things

a b s t r a c t In this paper, we propose a new congestion control algorithm called CoCoA++ to address the issue of network congestion in Internet of Things (IoT). Unlike the existing congestion control mechanisms that operate on instantaneous Round Trip Time (RTT) measurements in IoT, we use delay gradients to get a better measure of network congestion, and implement a probabilistic backoff to deal with congestion. We integrate the delay gradients and the probability backoff factor with Constrained Application Protocol (CoAP). The proposed algorithm is implemented and evaluated using the Cooja network simulator provided by Contiki OS. Subsequently, it is deployed and evaluated in a real testbed by using the FIT/IoT-LAB. We observe that delay gradients give a more accurate measure of congestion and the Retransmission Time Out (RTO) is reduced significantly, thereby leading to less delays and high packet sending rates. CoCoA++ being a minor improvement over the existing algorithm is easy to deploy. © 2019 Elsevier B.V. All rights reserved.

1. Introduction The advent of the Internet of Things (IoT) and its inclusion in everyday scenarios brings with it a fair share of problems. A large number of IoT devices are characterized by small memory and low processing speeds that lead to congestion in the network when many such devices try to communicate with each other. The data packets being shared by the constituent devices across the network in IoT systems have small payloads and hence, packet loss due to congestion results in expensive retransmissions that lead to additional delays and large overheads. IoT networks also have a mobility factor associated with them as they do not always deal with static nodes. The small packet size, costly re-transmissions and mobility together make the nature of these constrained networks very different from the conventional networks currently in place [1]. ∗ Corresponding author. E-mail addresses: [email protected] (V. Rathod), [email protected] (N. Jeppu), [email protected] (S. Sastry), [email protected] (S. Singala), [email protected] (M.P. Tahiliani). https://doi.org/10.1016/j.future.2019.04.054 0167-739X/© 2019 Elsevier B.V. All rights reserved.

Several mechanisms have been designed that use different aspects of the network to control congestion, and either actively or passively deal with it [2]. Packet loss is one of the most commonly used signs of congestion. Once a source sends a packet, it waits for a fixed amount of time for an acknowledgement from the destination by maintaining a timer at its end. If no acknowledgement is received within this duration, the packet is considered lost due to network congestion. Once a packet loss is detected, the congestion control algorithm in place takes charge to reduce the network congestion. Mechanisms that depend on packet loss to detect network congestion prove to be of no use in IoT networks because: (i) these mechanisms assume that a packet loss happens only due to congestion, which is not true for low power and lossy networks e.g., packet loss can happen due to link errors or poor signal strength [3], and (ii) delaying congestion response until a packet is lost deteriorates the performance since IoT packets usually carry small payloads and their retransmission is costly, causing additional delays [3]. Typically, the congestion control mechanisms are tightly coupled with TCP. The major concern is that TCP is not a defacto transport protocol for IoT

1054

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

applications [4], and hence, designing congestion control mechanisms for IoT applications is a non-trivial task. The congestion control mechanism proposed in this paper is suitable for a few UDP based IoT applications. Some congestion control algorithms use Round Trip Time (RTT) as a measure of congestion [5,6]. An increase in the RTT values is considered as an indication of congestion building up in the network. These algorithms compare RTT with a threshold value and trigger the congestion control mechanism if the measured RTT exceeds the threshold. However, RTT is not a reliable metric to predict network congestion because multiple factors, such as the processing time at the endpoints or mobility of nodes, could lead to an increase/decrease in RTT. Finally, fairness issues arise when RTT based congestion control algorithms share bandwidth with loss based algorithms. In such shared systems, a backoff by RTT based algorithms may cause the loss based algorithms to take up the freed bandwidth and the RTT based algorithms may never get a chance to transmit messages [7–9]. Therefore, it is necessary to make sure that the congestion control algorithm in place is able to detect the presence of such loss based algorithms sharing the network bandwidth. Constrained Application Protocol (CoAP) (RFC 7252) is one of the most widely used IoT protocols. Congestion control mechanisms that operate over CoAP, like CoAP Simple Congestion Control/Advanced (CoCoA) [6] and its variants [10], use per packet RTT measurements to predict congestion in the network. RTT measurements are typically noisy and cannot be relied upon for predicting network congestion. Hence, in this paper, we propose a congestion control mechanism for CoAP that relies on CAIA Delay-Gradient (CDG) [11] to predict network congestion and a Probabilistic Backoff Factor (PBF) to control network congestion. CDG provides a better measure of network congestion by obtaining a gradient of RTT over time [11] and PBF helps to adjust the RTO based on the inferred congestion [11]. We integrate CDG and PBF with CoCoA+ [12] in order to resolve the issues mentioned above, and name the resulting algorithm as CoCoA++. Since CoCoA++ is a minor enhancement over the existing congestion control algorithm for CoAP, its deployment complexity is low. Our results from Cooja simulations and real testbed at FIT/IoTLAB show a marked improvement in estimating congestion in the network, which is evident from the lower Retransmission Time Outs (RTO) and higher packet sending rates as compared to CoCoA+. The congestion control mechanism in Contiki OS [13] is referred to as CoCoA although it implements CoCoA+. Hence, for the sake of simplicity, we have also used the same terminology throughout this paper. It must be noted that, in this paper, we focus on congestion control mechanisms for CoAP alone. The congestion control algorithms for other IoT protocols like Message Queuing Telemetry Transport (MQTT) [14] are beyond the scope of this paper. The paper is organized as follows: Section 2 gives a background of the existing congestion control mechanisms for CoAP, specifically, CoCoA and CoCoA+ and discusses the related work on Congestion Control in IoT. Section 3 describes the concept of CDG and explains the proposed solution along with the implementation details. Section 4 presents the evaluation scenarios along with results obtained from Cooja simulations and FIT/IoTLAB testbed. The conclusion of the paper is given in Section 5 with potential future directions. 2. Background and related work 2.1. Background of CoCoA and CoCoA+ CoAP is primarily a UDP based protocol for constrained networks despite the fact that it is inspired by HTTP which is based

on TCP [15]. The main reason behind using CoAP over UDP is its simplicity and small message size [16]. A limitation of using CoAP over UDP is that some middleboxes (e.g., firewalls) block UDP packets and obstruct the integration of CoAP with existing Internet infrastructure [3]. This has encouraged the developers to consider deploying CoAP over low footprint TCP for constrained devices [3,17,18]. However, deploying a low footprint TCP still remains a major challenge because there is a trade-off between the complexity of its implementation and the performance benefits it provides. Furthermore, the results from an experimental study in [3] show that CoCoA RTO outperforms TCP RTO in terms of network wide throughput, especially in bursty traffic conditions. Due to these reasons, RFC 8323 notes that CoAP over UDP is still the recommended transport for constrained networks, and that CoAP over TCP is applicable for those cases where the networking infrastructure leaves no other choice. CoAP functions in between the application and transport layers in an abstract layer consisting of 2 logical sub-layers: messaging and request/response [19]. The messaging layer handles the asynchronous behaviour of UDP and the request/response layer provides message and response codes to handle different types of messages. CoAP being a UDP based protocol, provides optional reliability features with the help of confirmable and nonconfirmable message tags. The confirmable message tags are used when the sender expects an acknowledgement of data delivery from the receiver, otherwise non-confirmable message tags are used. Congestion control in CoAP is achieved by using confirmable messages that provide a means to track the state of congestion in the network. The default congestion control mechanism for CoAP uses two parameters to implement Binary Exponential Backoff (BEB) at the client side [19]: Timeout and RetransmissionCounter. Another variable AckTimeout, normally set to 2 s, is used to obtain an estimate about how much time the client should wait until it declares a transmission as unsuccessful. Initially, the timeout is set to a value between AckTimeout and AckTimeout multiplied by a AckRandomFactor, and RetransmissionCounter is set to 0. After a timeout, the message is re-transmitted. RetransmissionCounter is incremented by 1 and Timeout is doubled (RFC 7252). This continues either till a RESET /Response is received or MaxRetransmit value is reached. When a RESET message or MaxRetransmit is reached, the application is informed of failure. In the case where the client receives an acknowledgement/response, transmission is considered successful [10]. CoCoA is an end-to-end congestion control mechanism for CoAP, and an improvement over the default congestion control in CoAP [15]. In contrast to the congestion control in CoAP that keeps a fixed value for Timeout, CoCoA makes an estimate of the RTO based on the RTT values. Moreover, the design of congestion control in CoAP makes it respond in the same manner for different scenarios irrespective of the number of nodes, degree of congestion or other network parameters. RTO estimation in CoCoA takes into account the current state of the network and hence, gives a more adaptive congestion control mechanism for CoAP. Initially, the CoCoA algorithm makes a blind RTO estimate by taking into consideration the number of simultaneous CoAP transactions (denoted by NSTART ) that are created to one destination endpoint, as shown in Eq. (1). The default value of NSTART suggested in RFC 7252 is 1. blindRTO = 2 × NSTART

(1)

As messages are sent and received across the network, this RTO estimate is updated based on the RTT calculations at the end points. CoCoA leverages the Karn/Partridge algorithm (used by TCP for measuring RTT samples) [20] for obtaining RTT samples. However, CoCoA does not ignore the RTT samples of all

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

retransmitted packets as recommended in the Karn/Partridge algorithm. As a result, it maintains two separate estimates of RTO, namely: Strong estimate and Weak estimate. Strong estimate is maintained for transactions that are successful in a single transmission whereas weak estimate is maintained for transactions that suffered from re-transmissions. Nevertheless, for weak estimate, only the RTT samples of the first transmission and the first retransmission are considered. RTT samples of subsequent re-transmissions, if any, are ignored because when an acknowledgement arrives, it is not possible to associate it with first re-transmission or second re-transmission. Lastly, the overall RTO estimate maintained by CoCoA is updated each time when there is a change in the strong estimate or the weak estimate. CoCoA uses the TCP’s standard algorithm specified in RFC 6298 to estimate strong and weak RTO. RFC 6298 recommends that the RTO be estimated based on Eq. (2), where x represents strong or weak accordingly. First, for a newly measured RTT sample RTTx_new , the weighted average RTTx is updated by using a weight constant, α (= 41 as recommended in RFC 6298). Second, the difference between RTTx and RTTx_new is considered for obtaining a weighted average of RTT variation, RTTVARx by using a weight constant β (= 18 as recommended in RFC 6298). Finally, the estimated RTO, RTOx is obtained by adding RTTx and Kx times RTTVARx , where Kstrong = 4 and Kweak = 1 as suggested in the Internet draft of CoCoA. CoCoA sets Kstrong to 4, similar to what is used by TCP for estimating its RTO (RFC 6298), because RTTstrong does not account for retransmitted packets. CoCoA sets Kweak to 1 because RTTweak accounts for retransmitted packets and could lead to inherent ambiguity due to inaccuracy in RTT samples. (2)

RTOx = RTTx + Kx × RTTVARx The overall RTO for CoCoA is then calculated as a weighted average of the strong or weak RTO estimate and the previous overall RTO as shown in Eq. (3). The Internet draft of CoCoA suggests that an equal weight should be assigned to the current RTO estimate (strong or weak) and the previous RTO. RTO = 0.5 × RTOx + 0.5 × RTO

(3)

CoCoA+ is an extension of CoCoA with three major enhancements [12]: (i) it modifies the Eq. (3) in case of RTOweak by assigning less weight to RTOweak and more weight to previous RTO. This is to ensure that inaccurate RTT measurements done in RTOweak do not largely affect the overall RTO. No changes are made to Eq. (3) in case of RTOstrong . Eq. (4) represents the Eq. (3) after modifications suggested in the Internet draft of CoCoA+. (ii) it adds a RTO ageing mechanism to CoCoA. When there are idle periods (i.e., no transmissions occur), RTO will not be updated to track the current state of congestion in the network. To ensure that RTO does not contain stale congestion information, CoCoA+ resets RTO to 2 s if there is an idle period of 30 s, as recommended in RFC 7252. (iii) it replaces the Binary Exponential Backoff (BEB) by a Variable Backoff Factor (VBF).1 In case of packet drops, BEB doubles the RTO whereas VBF offers three different backoff factors, as depicted in Eq. (5). RTO = 0.25 × RTOweak + 0.75 × RTO RTO = 0.5 × RTOstrong + 0.5 × RTO

⎧ RTO < 1 s ⎨3, VBF = 1.5, RTO > 3 s ⎩ 2

As mentioned in the Internet draft of CoCoA [21], the RTO for a retransmission is multiplied by 3 for each retransmission as long as the RTO < 1 s. This is to ensure that transactions with small initial RTOs do not use up all retransmissions (MAX _RETRANSMIT ) in a short period of time. Similarly, the RTO for a retransmission is multiplied by 1.5 for each retransmission when RTO > 3 s. This is to ensure that transactions with large initial RTOs do not fail to carry out sufficient retransmissions before giving up on receiving an acknowledgement.2 In other cases, RTO is multiplied by 2 like BEB. CoCoA and CoCoA+ require the end points to maintain the strong and weak RTO estimates at all times. As RTO estimates are made using per packet RTT samples that are noisy, it may not provide a good measure of network congestion. CDG provides a better estimate of network congestion as it primarily attempts to track the changes in the queuing delay component of RTT, and ignores other delay components such as processing, propagation and transmission delays. Instead of considering per packet RTT samples, CDG tracks minimum and maximum RTT for a fixed period of time and repeats this process for several intervals. It compares the minimum and maximum RTT obtained at different intervals to infer the increase/decrease in queue delays e.g., if the minimum RTT does not change for several intervals but maximum RTT keeps falling, it indicates that the queue delay is reducing. This approach minimizes the noise caused by other types of delays and provides a much better congestion signal. More details about CDG are provided in Section 3. 2.2. Related work

RTTx = (1 − α ) × RTTx + α × RTTx_new RTTVARx = (1 − β ) × RTTVARx + β × (RTTx − RTTx_new )

1055

(4)

(5)

Otherwise

1 Although the CoCoA paper [10] contains discussions on using VBF instead of BEB, the latest paper on CoCoA+ from same authors states VBF as one of the features added in CoCoA+ [12]. Hence, in this paper, we treat VBF as a feature introduced in CoCoA+.

The literature on congestion control in IoT spans across different layers of the network stack. In this section, we focus on the recent developments directly related to our work i.e., application level solutions to control network congestion in IoT. We also briefly discuss a few routing level solutions for network congestion control. 2.2.1. Application level solutions CoAP is a standard application protocol used in IoT and it is mainly designed to run atop UDP. There has been a lot of interest in designing congestion control algorithms for CoAP. A comprehensive survey of the congestion control mechanisms designed for CoAP is presented in [22]. Majority of the solutions include modifying the default RTO estimation technique used by CoAP. These solutions improve the performance of CoAP/CoCoA, but do not extract a clear congestion signal from noisy RTT samples. Several studies have focused on evaluating the performance of CoCoA in a wide variety of scenarios ranging from emulated Zigbee networks to large scale IoT deployments [19,23]. Some of these studies have shown that CoCoA provides better performance than CoAP [24–26], whereas others have discussed the shortcomings of CoCoA e.g., [27] shows that CoCoA has more retransmissions than CoAP when the number of requests increase. A new method is proposed in [28] to calculate RTO in CoAP based on the experimental study of Eifel retransmission timer [29], which is originally proposed for calculating TCP timeouts. During this study, the authors observed that the default values of α ( 14 ), β ( 18 ) and K (4) suggested in RFC 6298 might not be suitable for large senders load. Subsequently, they propose a new RTO estimation algorithm that uses a single coefficient γ instead of α and β . γ is defined as a ratio between current sample of RTT and RTO. Depending on the value of γ , the authors suggest to carefully adapt RTO. However, the discussion on extracting 2 RFC 7252 specifies 93s from the first transmission to the time after which the sender gives up on receiving an acknowledgement.

1056

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

congestion signals from the RTT samples is missing. CoCoA++ algorithm proposed in this paper provides a simpler solution to obtain reliable congestion signals by using CDG. Four modifications of CoCoA are discussed in [30]: CoCoA-Fast (CoCoA-F), CoCoA-Strong (CoCoA-S), CoCoA-4-State and CoCoA4-State-Strong. This work mainly highlights that the performance of CoCoA is conservative in the presence of lossy wireless links, owing to the side effect of its weak estimator. Initially, the authors try to increase the aggressiveness of CoCoA by reducing the values of VBF, backoff thresholds, initial and maximum RTO, and name the resulting variant as CoCoA-F. However, it is observed that the performance of CoCoA-F remains conservative like CoCoA. Subsequently, the authors propose a 4-state higher granularity estimator that helps to distinguish between wireless link losses and congestion losses. Finally, three schemes: CoCoAStrong, CoCoA-4-State and CoCoA-4-State-Strong are discussed to overcome the performance problems of CoCoA in lossy wireless links. The results show that CoCoA-4-State-Strong adapts to packet losses and achieves 35%–60% more throughput than CoCoA. However, it also leads to 20% more retransmissions than CoCoA. An adaptive mechanism for handling congestion in CoAP is proposed in [31] that depends on traffic priority and considers the packet loss rate. This mechanism has three main components: assigning traffic priority, adaptive RTO and adaptive backoff timer. Priorities are assigned depending on whether the traffic emerges from a critical device (e.g., medical sensors or fire monitoring systems) or a non-critical device (e.g., smart home devices). Depending on traffic priorities and the packet loss rate, RTO and backoff timer values are calculated. The major limitation of this approach is that it is not implemented and evaluated in the paper which raises concerns about the feasibility of its deployment. Yet another adaptive congestion control scheme for CoAP is proposed in [32] which relies on accurately estimating the RTT of retransmitted packets, precisely when a single packet needs multiple retransmissions (i.e., when retransmissions fail). The authors show that this approach offers improvements in terms of throughput and number of successful transactions when the number of clients is high (i.e., when network is congested), but otherwise performs similar to CoAP. However, this approach is not effective when a packet does not need multiple retransmissions (i.e., when a single retransmission suffices). Real-time IoT traffic is diverse in nature. Hence, a thorough study is presented [25] to evaluate the performance of CoCoA by considering typical IoT scenarios with different traffic patterns. CoAP and CoCoA have been assessed by means of simulations with various different loads that have been offered as per the increasing number of clients. The performance of CoCoA is worse than that of CoAP when the number of clients are less and with bursty traffic performance; it worsens further. It was observed that the performance of CoCoA was poor for bursty traffic patterns due to inappropriate RTO estimation. We evaluate CoCoA++ in scenarios with different traffic patterns in Section 4.2.1, and observe that its performance is robust against the varying traffic patterns. CoAP with a new rate based congestion control (CoAP-R) [33] is yet another algorithm targeted to improve the performance of CoAP in bursty traffic environments. CoAP-R controls the sending rate of CoAP sources and endorses rate-based mechanism (instead of window based mechanisms) to control the traffic. CoAP-R is designed to achieve maximum bandwidth from the bottleneck link capacity and allocate the network resources based on the max–min fairness. The simulation results in the paper show that CoAP-R distributes the network resources equally amongst the senders and reduces the delay compared to CoAP and CoCoA. Depending on extensive evaluation, [34] highlights the shortcomings of CoCoA+: RTO too close to RTT, lack of weak estimator

update, insufficient weak estimator weight, RTO peaks in response to RTT decrease and excessive RTO growth. This work also makes an observation about the poor performance of CoCoA in bursty traffic patterns. To address these limitations, a new extension called precise CoCoA (pCoCoA) is proposed and evaluated. pCoCoA eliminates the use of weak estimator, introduces a new way to initialize RTT measurement parameters and reduces spurious retransmissions. The results show that pCoCoA reduces the number of retransmissions without affecting the throughput and delay. However, some of the fixed values introduced in the paper might not be suitable for a wide range of IoT scenarios. IoT plays a crucial role in health-related applications. A comparative study of CoAP and CoCoA in e-health scenarios is presented in [26]. The authors evaluate the performance of CoAP and CoCoA in different topologies such as single client to single server, multiple clients to single server, single client to multiple servers and multiple clients to multiple servers. The parameters considered for comparison are goodput, packet delivery ratio, average delay and total number of packets dropped. It is observed that the total number of packets dropped in CoCoA is much lesser than CoAP. Also, CoCoA outperforms CoAP in terms of goodput in all topologies. The authors also show that CoCoA consumes less bandwidth, is more adaptive and scalable than CoAP. Bufferbloat refers to the problem of having excessive queue delays in the network. In [35], the authors state that different congestion control mechanisms designed for CoAP fail to handle the problem of Bufferbloat and perform spurious retransmissions. Through extensive evaluations, the authors show that CoAP and CoCoA fail to handle the problem of Bufferbloat, and due to this, there is significant wastage of the network bandwidth. Moreover, authors also find the main cause of inaccuracy in the backoff logic RTO. Subsequently, the RTO backoff is modified to overcome this concern. Along similar lines, [36], attempts to overcome the problem of bufferbloat in scenarios that are prone to heavy congestion. A new mechanism called Fast–Slow RTO (FASOR) is proposed which tries to infer whether the packet loss is due to the wireless link disruption or due to congestion. It implements the following three fundamental functionalities: Fast RTO computation, Slow RTO computation, and Self-adaptive retransmission timer backoff mechanism. Fast RTO computation is analogous to the TCP RTO mechanism except the minimum RTO bound, and it is evaluated for explicit RTO samples. Slow RTO’s functionalities are similar to those of Karn’s algorithm. Its usage is to keep minimal time for backoff. The results show that FASOR has shorter Flow Completion Times (FCT) compared to CoAP and CoCoA, and it controls the RTO in heavily congested scenarios. Evaluating a new algorithm in large-scale IoT deployment is a non-trivial task. In [37], the authors present an evaluation of default congestion control mechanism of CoAP in real time experiments with the WiSHFUL platform (Wireless Software and Hardware platforms for Flexible and Unified radio and network ControL) [38]. WiSHFUL is a large-scale experimental platform which offers a unique interface to design and control the experiment at run-time. The main goal of this work was to verify the working of WiSHFUL platform, and as a part of a case study, the default CoAP congestion control is compared with a simple algorithm based on the measured RTT value over time. The results show that the default congestion control mechanism is more scalable and robust than the simple algorithm. In [39] a new feature called observe is proposed which provides information about the state of a resource at CoAP server to a CoAP client. The authors conduct experiments and highlight the need to have congestion control at both CoAP clients and servers, particularly when features like observe are deployed. It is mentioned that different RTT estimation algorithms can be used at clients and servers but there are no recommendations about the choice of congestion control algorithms that are most suitable.

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

2.2.2. Routing level solutions Another approach of controlling network congestion is by designing load-aware routing protocols. It is done by making all communications follow a hop-by-hop approach, but this significantly increases the control overhead in the network as each node has to share the measured congestion level with other nodes. An analysis of congestion in 6LoWPAN (IPv6 over Low-Power Wireless Personal Area Networks) networks shows that a large number of packet drops at routing layer are due to buffer overflow rather than channel loss [40]. So, [41] proposed a new routing metric for IPv6 Routing Protocol for Low-Power and Lossy Networks (RPL) which prohibited the number of packet losses due to buffer overflow when congestion occurred. They came up with a new routing metric called Buffer Occupancy (BO) and a new objective function called Congestion-Aware Objective Function (CA-OF). The results show improvement in the throughput and packet delivery ratio. Along the same lines, [40] proposed that buffer occupancy should be used as a measure of congestion, and accordingly proposed a new version of RPL. Such an approach requires extra control messages to exchange the buffer occupancy information among neighbour nodes, and hence incurs more overhead. Most of the packet losses in heavy traffic scenarios are due to congestion and load balancing problems that emerge during the parent selection in routing [42,43]. In order to tackle this issue, a new mechanism called simple Queue Utilization based RPL (QURPL) has been proposed in [42]. In QU-RPL, parent selection is done based on queue utilization of the neighbour nodes as well as hop distances from RPL border router. This mechanism is very adequate in heavy queue losses and increases the packet delivery ratio. In [44], the authors note that when transmission power and routing topology are controlled collectively and attentively, multihop wireless network gains better bandwidth and stability. There is a significant loss in bandwidth due to the construction of routing topology (based on link quality and hop distance) with fluctuation in transmission power. So, a robust and dispersed mechanism called Power Controlled RPL (PC-RPL) is proposed. It is a control mechanism which handles transmission power and routing topology efficiently. In [45], to understand the trade-offs in choosing TCP over RPL, an empirical study has been done on it. Their results show how embedded TCP interoperates with Linux TCP and underlying RPL (and vice versa). 3. Proposed methodology

1057

where, g is the delay gradient, gi = gmin,i for calculating g¯min,n or gi = gmax,i when calculating g¯max,n , g¯n represents the average of gradients and a is the number of samples in the moving average window. The rate of change of RTT is used to calculate a probability of backoff (as shown in Eq. (7)). The probability approaches 1 as the rate increases. The probability is then compared with a random value to determine whether to reduce the congestion window of TCP (by a factor of 0.3 as suggested in [11]). When it decides not to reduce congestion window, it is increased by 1 [11]. P [backoff ] = 1 − exp−(

g¯n ) G

(7)

where, G > 0 is a scaling parameter used to control the probability of backoff due to delay gradient indications [46]. The decision to reduce TCP congestion window is taken not only based on the probability but also by differentiating between packet loss caused due to congestion and packet loss caused because of other factors. In general, congestion occurs when the queue becomes full. CDG estimates this based on the RTTmin and RTTmax values. If RTTmax reaches its largest possible value and stops increasing and if RTTmin is still increasing, it means that the queue is full and any packet loss during this time is because of congestion. Hence, CDG updates the congestion window (W ), as per Eq. (8) [11]:

{ Wn =

Wn−1 × 0.7, P[backoff] > X and gmin > 0 Wn−1 + 1,

otherwise

(8)

where, Wn indicates the congestion window during the nth interval and X is a uniformly distributed random number, X = [0, 1]. The advantage of using CDG over other congestion indicators is that it is able to handle shared systems that use loss-based congestion control. When a CDG connection shares bandwidth with a loss based system, in the event of congestion building in the network, CDG will back off. This gives the loss based system an opportunity to continuously pump packets and take up the entire bandwidth. To deal with this, the CDG algorithm keeps track of how the RTT values change after the back off. The correct functioning of CDG demands that a decision to slow down must result in decreasing RTT values (gmin and gmax become negative). If this is not observed, it is understood that CDG is competing with a loss based system and the algorithm stops backoff. CDG also stores the previous value of maximum congestion window which can be easily restored in case of a packet loss.

3.1. Overview of CAIA Delay-Gradient (CDG) CDG was proposed to work alongside TCP to provide better inferences of network congestion. However, CDG can be integrated with any other end to end protocol (e.g., CoAP in our case) since it is not tightly coupled with TCP. This section explains the working of CDG with TCP to control network congestion. In the subsequent section, we provide the details of integrating CDG with CoCoA+ to control network congestion in IoT. TCP congestion control algorithms that use CDG try to predict congestion by tracking minimum and maximum RTT for a fixed period of time (5 s as recommended in [11]) for several intervals (denoted by n) [11]. The gradients (g) track the rate of change of RTTmax and RTTmin values to determine if the network is congested. Eq. (6) shows the calculation of delay gradients: gmin,n = RTTmin,n − RTTmin,n−1 gmax,n = RTTmax,n − RTTmax,n−1 g¯n =

n ∑ gi i=n−a

a

(6)

3.2. Design of CoCoA++ CoCoA++ is a congestion control mechanism for CoAP that integrates CDG with CoCoA+. The main goal of the proposed algorithm is to provide better estimate of network congestion to end points by leveraging the advantages of CDG. However, integrating CDG with CoCoA+ is a non-trivial task because the design of CDG is tailored to work alongside TCP. CDG provides information about network congestion which is then used by TCP to update the congestion window. The major concern is that the concept of congestion window does not exist in the CoCoA+ protocol, which makes it difficult to seamlessly integrate CDG with CoCoA+. This issue is tackled in CoCoA++ by using the information obtained from CDG to derive better RTO estimate. CoCoA++ eliminates the need to maintain two RTO estimates i.e., strong and weak RTO estimates. Moreover, CoCoA++ does not update the RTO based on per packet RTT samples like in the case of CoCoA and CoCoA+. Instead, it updates RTO when periodic information about

1058

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

the delay gradients gmin and gmax is received from CDG.3 As a consequence, CoCoA++ does not rely on VBF for backoff since VBF assumes that RTT is calculated for every packet. CoCoA++ replaces VBF by a Probabilistic Backoff Factor (PBF) which is calculated as shown in Eq. (9). 1.42,

P[backoff] > X and g > 0

0.7,

otherwise

{ PBF =

3.3. Implementation of CoCoA++ We implemented CoCoA++ in Contiki-3.0, simulated it using the Cooja network simulator [13] and tested its performance in a real testbed at FIT/IoT-LAB [48]. Modifications were made to the existing CoCoA implementation5 to integrate CDG and PBF. All experiments use the default Contiki OS stack for implementation and evaluation. Our implementation of CoCoA++ uses a lookup g¯n ) G

RAM ROM MCU Radio

T-mote Sky

Z1

10 kB 48 kB MSP430F1611 CC2420

8 kB 92 kB MSP430F2617 –

(9)

The probability of backoff obtained from CDG (Eq. (7)) is compared with a uniform random value X to ensure that the flows with small RTT and those with large RTT have a similar probability of backoff. The condition g > 0 ensures that the backoff is applied to RTO only when the rate of change of RTT is positive. When congestion is detected in the network (P [backoff ] > X and g > 0), the RTO is increased by a factor of 1.42. RTO is reduced by a factor of 0.7 when no congestion is detected. The value 0.7 to reduce RTO is obtained by performing simulations with a range of values from [0.5, 1.0].4 This range is considered with a goal to ensure that RTO value reduces gradually, and that this reduction does not lead to congestion in the network. It was observed that the best performance is obtained with 0.7. Accordingly, its reciprocal (i.e., 1.42) was selected to increase the value of RTO when the network is congested. The overall working of CoCoA++ algorithm is summarized in Fig. 1.

table for exp−(

Table 1 Sensor mote configuration.

which is available in Eq. (7).

4. Evaluation of CoCoA++ In this section, we discuss the methodologies used to evaluate the performance of CoCoA++ and compare it with CoCoA.6 The evaluation is carried out in two ways: (i) by configuring static and mobile topologies in Cooja simulations, and (ii) by configuring a real testbed at FIT/IoT-LAB [48]. In addition, we extensively evaluate both the algorithms with different traffic rates in Cooja and testbed. We track how the RTO values change with time, and measure the average number of transactions per second. 4.1. Simulation setup From the available mote types in Cooja, Zolerita Z1 motes [49] are used for clients and servers, with transmission and interference ranges set to 10m and 20m, respectively. Z1 motes have large capacity of ROM that enables us to code applications and implement congestion control mechanisms. T-mote [50] Sky 3 At most once every two RTT intervals as mentioned in [11]. Although designed to work with TCP, this period is also suitable for CoCoA++ because RTO calculations in CoCoA++ are not significantly different than in TCP. Our results provided in [47] confirm that this period works well with CoCoA++. 4 Results obtained from simulations with these values are provided in [47]. 5 Thanks to authors of CoCoA for sharing its code for Contiki OS. 6 The enhancements proposed in CoCoA+ are incorporated in Internet draft of CoCoA. Moreover, Contiki OS also refers to it as CoCoA instead of CoCoA+. Henceforth, we have used the term CoCoA instead of CoCoA+.

motes are used to configure the border router because we use IPv6 Routing Protocol for Low-Power and Lossy Networks (RPL) in storing mode, and this requires larger RAM capacity which is provided by Sky motes. Table 1 shows the configuration of T-mote Sky and Zolerita Z1 motes in terms of RAM, ROM, Microcontroller Unit (MCU) and Radio transceiver. After the RPL setup, the clients periodically send messages targeted at the servers. Every scenario is run 30 times with a different random seed. The simulation results for all runs are averaged and plotted against the simulation time. Initially, we evaluate the performance of CoCoA++ in static and mobile topologies, which is followed by evaluation with different traffic rates. 4.1.1. Static and mobile topologies 1. Static topologies: The nodes have fixed positions and zero mobility throughout the simulation. Four different topologies are considered: grid, flower, dumbbell and chain. Grid, dumbbell and chain topologies are analogous to the ones used for evaluating CoCoA and CoCoA+ [12]. Grid topology: Grid topologies are popular in applications requiring long range and broad area coverage such as smart grid, industrial automation and building automation [6,12, 25]. They are designed in such a way that every node (a sensor or router node) has at least one other node within its range of transmission. Data packets are relayed across several nodes before they reach the server, and hence the network range is independent of the transmission range of the individual nodes. In this topology, the internal nodes of the grid, or in other words, nodes that are adjacent to four neighbours, act as the potential points of congestion. The grid topology we use is a 6 x 6 grid consisting of 2 servers, a border router and the clients, arranged in a grid where the nodes are spaced 10m from each other [See Fig. 2]. Flower topology: Flower topologies like the one illustrated in Fig. 2 are used in applications such as smart greenhouse management systems, cellular network, satellite network and wide area network applications [51]. These consist of a central processing system that controls environment parameters like water flow, temperature etc. for the confined space based on the information it receives from sensors placed in the individual pots. All sensor information is relayed across the array of potted plants that are represented by nodes in the branches of the flower topology. The central server acts as a bottleneck, and hence is a potential point of congestion. The flower topology we use consists of 36 nodes with a single server, one border router and 34 clients as shown Fig. 2. The communication from client to server (when not in range for direct communication) happens through the nodes arranged as petals. Dumbbell topology: It is the most widely used topology for evaluation of network congestion algorithms [6,12]. The link between the two halves of the dumbbell serves as the bottleneck. When the bandwidth offered by this link cannot accommodate all client transactions, the link needs to be shared in a fair manner which is ensured by the congestion control algorithm in place. The dumbbell

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 1. The working of CoCoA++ algorithm.

Fig. 2. Grid and Flower topologies.

1059

1060

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 3. Dumbbell and Chain topologies.

topology we use consists of 19 client nodes, one server and a border router arranged in the shape of a dumbbell with the border router at the centre as shown in Fig. 3. The set of clients forming one half of the dumbbell on the right side are within the range of the server, whereas any communication between the clients in the left half of the dumbbell and the server takes places through the border router. Chain topology: Networks using chain topology experience the least amount of congestion. The communication from one end to the other takes place across the chain as messages are relayed from a node to its immediate neighbour. The chain topology we use consists of 18 clients, a server and border router arranged in a line as shown in Fig. 3. 2. Mobile topologies: The nodes move during the simulation based on four different mobility models that exhibit the behaviour of mobile nodes in some IoT applications such as smart traffic management, self driving cars, home automation system with handheld devices and communication for security drones [52]. Note that in all our mobility simulations we keep the border router and servers stationary and only the clients are given some non-zero velocity. We have used BonnMotion: a mobility scenario generation and analysis tool for generating a position file of a particular mobility model [52]. Random Waypoint Mobility Model (RWMM) [53]: Random Waypoint is a mobility model where the nodes travel in a zig-zag manner. Initially the nodes remain stationary for some fixed amount of time during which they decide on a destination and a random speed within the feasible range. Once they reach their destinations, they again pause for some interval before repeating the process. The model is incorporated into the simulation using the mobility plugin for Cooja where it is possible to specify a position.dat file that contains the position of every node at different points in time. These positions are generated using the Random Waypoint Mobility model [54]. Manhattan Grid Mobility Model [55]: Manhattan mobility model is used widely in Vehicular Ad-hoc Networks. It is typically used to simulate the behaviour of vehicles on a grid road topology and guide them to take the right path. The nodes in this model move along the horizontal and vertical grid lines. At each intersection the nodes turn right, left or continue straight with some probability.

Pursue Mobility Model [53]: Pursue mobility model is designed to exhibit the behaviour of one node (pursuer) following another node (target). The target node moves across the simulation region based on Random Waypoint and velocity of the pursuer node is directed towards the target. Gauss–Markov Mobility Model: Gauss–Markov mobility model correlates the velocity of mobile node with time [54]. It is modelled as a Gauss–Markov stochastic process [56] where the velocity of the node at some time slot t depends on its velocity in the previous time slot t − 1. The model is a temporally dependent model with a parameter α that determines the degree of dependence. For zero dependence, α is set to 0 (no memory or memoryless model) and for very strong dependence the parameter α is set to 1 (strong memory model). 4.1.2. Varying traffic rates The scenarios used for evaluation in this section are identical to the ones mentioned in the previous section for static and mobile topologies, except that we repeat the experiments with different traffic rates. In these scenarios, each sensor periodically sends measurement reports to the sink node. Different traffic loads are generated by varying the frequency of reporting the sensor measurements. The main goal of this scenario is to evaluate the robustness of CoCoA++ against different traffic rates. The parameters used for simulations and their respective values are mentioned in Table 2. The values shown in Table 2 are same as the ones used for evaluating CoCoA+ in [12], except that we use mobility models like Random Way Point, Gauss–Markov, Manhattan Grid and Pursue to evaluate the performance of CoCoA++ in mobile IoT applications such as self-driving cars and drones. The parameter Radio medium indicates the propagation loss model used in the simulation. Radio duty cycling handles the sleep and wakes up cycle depending on the type of the RDC method used by the motes. Motes are configured without Radio Duty Cycling (NullRDC) [58], which means that the motes are always in listening mode and do not sleep. 4.2. Simulation results We evaluate and compare the performance of CoCoA and CoCoA++ under the same simulation setup by looking at the variation in RTO values with simulation time. RTO measurements

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

1061

Fig. 4. Comparison of CoCoA and CoCoA++ in static topologies.

give us an idea of how much delay is encountered in transmission, which indirectly gives us an estimate of packet sending rates. Each scenario is run 30 times with different seed values and the average RTO values are plotted against the simulation time [Figs. 4 and 5]. The plots represent the variations in RTO values with simulation time for both CoCoA and CoCoA++. Each point on the graph represents the RTO value for a transaction that is initiated at the time represented by the corresponding time on the simulation time axis. This gives us an idea of the number of transactions that take place in the given simulation interval and the distribution of RTO values. As shown in Table 3, the maximum

and average RTO values are calculated over all simulations for a particular scenario by averaging the values obtained in the all 30 simulations. From the plots and Tables 3 and 4; it is observed that in static scenarios, the RTO values obtained for CoCoA++ do not exceed 13 s, whereas the RTO values for CoCoA go up to 38 s, which is almost three times the maximum value obtained for CoCoA++. Similarly, with all the mobility models discussed above, the RTO values obtained for CoCoA++ do not exceed 20 s, whereas the RTO values for CoCoA go up to 35 s. On an average, the RTO measurements obtained for CoCoA++ are significantly lower

1062

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 5. Comparison of CoCoA and CoCoA++ in mobile topologies.

than CoCoA, thereby reducing the transmission delays. In IoT networks where messages comprise mostly of small payloads, a reduction in transmission delays can enhance performance by a large margin. Lower transmission delays also increase the number of transactions per second. As in shown Table 3, CoCoA++ records more than double the transaction rate as compared to CoCoA in static scenarios. Similarly, with all the mobility models discussed above, the transaction rates of CoCoA++ increase by nearly threefolds (Table 4). One of the many aspects that contribute to low RTO values and high packet transmission rates in CoCoA++ is the

use of PBF as shown in Eq. (9). The highest backoff factor in PBF (1.42) is smaller than the lowest backoff factor in VBF (1.5). The above observations can be justified based on the fact that delay gradients give a better estimate of congestion that is persistent in a network. By taking the rate of change of RTT rather than instantaneous values, we focus only on change in the queuing delay components and ignore other delay components like processing time at the endpoints that otherwise contribute to the RTT as well. Any congestion in the network will result in an

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 6. Comparison of CoCoA and CoCoA++ in terms of CDF. Table 2 Simulation parameters. Parameters

Values

Operating System Simulator Radio Medium Radio Duty Cycling Mote Type Number of Nodes Node transmission range Node Interference range Transmission/Reception ratio Traffic Type Simulation Duration Retransmission limit Mobility Model

Contiki-3.0 Cooja Unit Disk Graph Medium - Distance Loss (UDGM) NullRDC T-mote Sky and Zolerita Z1 Flower, Grid, Mobile topologies - 36, Dumbbell - 21 and Chain - 20 10 m 20 m 50% Periodic traffic from Client to Server, packet interval 0.4 s 18 min for each simulation 4 RandomWayPoint, Gauss–Markov, Pursue and ManhattanGrid

1063

1064

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 7. Grid topology.

Fig. 8. Flower topology.

increase in queuing delay which is captured by the delay gradient and hence, acts as a sign of true congestion. Fig. 4 presents the variations in RTO values with both CoCoA and CoCoA++ for static topologies. A point in the graph represents the RTO values with respective to the simulation time. RTO values

with CoCoA exhibit a linear incremental pattern followed by a sharp decrement, and this cycle repeats throughout the simulation. The noisy RTT signals in CoCoA lead to large variations in RTO values, making it to difficult for the algorithm to converge to a stable operating point. On the other hand, CDG filters out

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

1065

Fig. 9. Dumbbell topology.

Fig. 10. Chain topology.

the undesired delay signals and provides a clear information about network congestion. Hence, the RTO values with CoCoA++, after the initial ramp up period, remain stable throughout the simulation. In summary, the RTO values with CoCoA++ are more stable, uniformly distributed and lower than those with CoCoA

because the effect of the rate of change of RTT gives good control over the RTO values in CoCoA++. Due to these characteristics, CoCoA++ successfully completes more number of transactions in a unit time than CoCoA.

1066

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 11. Random Way Point mobility model.

Fig. 12. Gauss–Markov mobility model.

Fig. 5 presents the variations in RTO values with both CoCoA and CoCoA++ for mobile topologies. Node mobility makes it further difficult to infer congestion from RTT measurements because mobility leads to variations in the received signal strength. Due to this, the overall RTT varies, and subsequently affects the RTO

estimation. This is evident when we compare the RTO values with CoCoA in static topologies and mobile topologies, and observe that the RTO values with CoCoA are largely scattered in mobile environments. However, this is not the case with CoCoA++ because it does not directly depend on RTT measurements for

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

1067

Fig. 13. ManhattanGrid mobility model.

Fig. 14. Pursue mobility model.

RTO estimation, but instead uses CDG. Since CDG helps to detect network congestion in a reliable manner, the RTO values with CoCoA++ are not affected in mobile environments and continue to exhibit a stable behaviour.

We reproduce all the plots of CoCoA and CoCoA++ shown in Figs. 4 and 5 in terms of Cumulative Distribution Function (CDF) of the RTO values in Fig. 6. We choose this style of representing the results because the CDF plots provide a clear understanding

1068

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 15. Deployment of nodes in Lille, Paris and Grenoble site at FIT/IoT-LAB [57].

Table 3 Results with static topologies. Scenario (Static topologies) Grid Flower Dumbbell Chain

CoCoA

CoCoA++

Max RTO Avg RTO (s) (s)

Average Transactions per second

Max RTO Avg RTO Average Transactions (s) (s) per second

24.268 24.345 37.717 24.288

8.7388 10.2268 5.9268 3.7666

11.466 11.499 12.968 11.480

8.5789 6.9081 6.2547 8.4768

4.6999 4.6174 3.5933 4.6419

21.7555 19.9148 10.8953 9.3231

Table 4 Results with mobile topologies. Scenario (Mobile topologies) Random Waypoint Gauss–Markov Manhattan-Grid Pursue

CoCoA

CoCoA++

Max RTO Avg RTO (s) (s)

Average Transactions per second

Max RTO Avg RTO Average Transactions (s) (s) per second

35.498 27.992 26.084 30.803

8.9685 8.8675 8.9722 7.8296

19.209 11.506 18.438 13.584

8.5824 8.4312 8.4062 8.5731

4.4809 4.5830 4.4525 4.5615

22.4953 22.0685 22.9666 22.0712

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

1069

Fig. 16. Topology selection from Lille, Paris and Grenoble site of FIT/IoT-LAB [57].

about the effectiveness of CoCoA++ in terms of keeping the RTO values lower than CoCoA. 4.2.1. Varying traffic rates This section discusses the simulation results obtained with different traffic loads. The traffic load in the network is varied from 0.5 Kbps to 2.5 Kbps, with an increment of 0.5 Kbps. We evaluate the performance of CoCoA and CoCoA++ in terms of the average RTO, the total number of packets transmitted and average packet sending rate. 1. Static topologies: Figs. 7–10 show the results obtained in static topologies for CoCoA and CoCoA++ with different traffic rates. Increasing the traffic rate increases congestion in the network and accordingly, the RTO values for both algorithms increase. RTO values with CoCoA remain consistently higher than CoCoA++ in all the topologies even when the offered load is low. This is expected because CoCoA relies on RTT and does not estimate queue delay. Similarly, the total number of packets transmitted and the average packet sending rate of CoCoA++ are always higher than CoCoA. Although the margin of improvement differs depending on the respective topology, the performance of CoCoA++ is significantly better in all topologies. The probability backoff factor of CoCoA++ plays a vital role in these scenarios by appropriately estimating the RTO values. 2. Mobile topologies: Figs. 11–14 show the results obtained in mobile topologies for CoCoA and CoCoA++ with different traffic rates. In mobile topologies, the average RTO values are almost halved in CoCoA++ as compared to CoCoA.

The reason for this is that CoCoA falsely assumes that the increment in RTT is an implication of congestion, and accordingly increases its RTO values. CoCoA++ measures queuing delay precisely, and it does not increase its RTO inadequately. The average packet sending rate of CoCoA++ in mobile scenarios is also higher than CoCoA. CDG and PBF both play an important role in these scenarios. The retransmission timeout mechanism of CoCoA++ is highly dependent on the relative movement of RTT; thus the total number of packets sent by CoCoA++ are much higher than CoCoA. 4.3. Testbed setup There are several testbeds available for IoT experimentation, like FlockLab [59,60], FIT/IoT-LAB [48,61] and others. For comparing the performance of CoCoA++ with CoCoA we choose FIT/IoTLAB because of its scalability, functionality, and variety of nodes. FIT/IoT-LAB7 provides an extensive scalable framework to perform experimentation with tiny sensor devices and with complex communicating objects. This lab is a part of the Future Internet of Things (FIT) platform. It gives full control to the researchers for accessing the network nodes as well as the gateways to which the different nodes are connected. FIT/IoT-LAB allows monitoring of the various network related performance metrics like throughput, network overhead, power consumption of nodes and others. The lab offers quick deployment of experiments, along with the ease 7 https://www.iot-lab.info/.

1070

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

Fig. 17. Comparison of performance metric in Lille, Paris and Grenoble site at FIT/IoT-LAB.

of evaluation, results collection and analysis [61]. The only major limitation of this lab is that it does not provide adequate support to perform mobile IoT experiments. The testbed contains various architectures which are distributed at six different locations across France: Grenoble, Lille, Saclay, Strasbourg, Paris, and Lyon. FIT/IoT-LAB uses specially designed boards for the testbed, e.g. WSN430 node, M3 node, A8 node. In our experiments, we use the M3 open node which is based on an STM32 (ARM Cortex M3) micro-controller. M3 open node supports FreeRTOS, Contiki and RIOT operating systems. Out of the six different locations of FIT/IoT-LAB, we choose three locations for our experiments: Lille, Paris and Grenoble (See Fig. 15). Lille, Paris and Grenoble are the most active locations of FIT/IoT-LAB that support 640, 332 and 160 wireless sensor nodes, respectively. They also support large number of M3 open nodes. Fig. 15 shows the real setup of the wireless sensor nodes in each particular site. We configure a grid topology in our testbed because the internal nodes in grid witness more congestion than any other nodes,

and such a configuration would help to evaluate the effectiveness of CoCoA++ in heavily congested networks. Fig. 16 shows the deployment of M3 open nodes and the selection of grid topology in each location. We have selected 9 (3*3) M3 open nodes at Lille and 6 (3*2) at Paris and Grenoble for our experiments. 4.4. Testbed results We configure five different traffic loads for evaluating the performance of CoCoA and CoCoA++ in FIT/IoT-LAB. The traffic load has been varied from 0.5 Kbps to 2.5 Kbps with an increment of 0.5 Kbps. We have used two performance metrics for evaluation: total number of packets transmitted and average packet sending rate. As shown in Fig. 17, the number of packets transmitted by CoCoA++ is much higher than CoCoA in all three sites of the FIT/IoT-LAB. CoCoA++ uses CDG, and its RTO estimation depends on the relative movement of RTT; hence it can transmit more packets than CoCoA. The average packet sending rate of CoCoA++ is also better than CoCoA.

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

5. Conclusions and future work In this paper, we proposed a congestion control mechanism called CoCoA++ that uses delay gradients to estimate network congestion in CoAP based IoT networks. We show that congestion control algorithms that solely depend on per packet RTT measurements, such as CoCoA, do not perform well due to the inability to infer network congestion from noisy RTT samples. Since delay gradients provide a fairly accurate estimate of network congestion we integrated CAIA Delay-Gradients and a Probability Backoff Factor with CoCoA as an enhancement over the existing algorithm implemented in Contiki OS. We evaluated CoCoA++ in static and mobile topologies to obtain a better understanding of how the algorithm fares in various IoT scenarios. The results show that CoCoA++ has low RTO values, thereby reducing transmission delays and increasing the transmission rate. In addition, we evaluated the functionality of CoCoA++ with different traffic rates and noted that it provides the desired performance. Depending on the inferences made from the simulations results, we tested the effectiveness of CoCoA++ by deploying it in a real testbed at FIT/IoT-LAB. We evaluated CoCoA++ with different workloads of a real-time IoT environment at three different locations of FIT/IoT-LAB: Lille, Paris and Grenoble. The results show that the total number of packets transmitted and average packet sending rate with CoCoA++ is higher than CoCoA. We believe that CoCoA++ is a promising congestion control algorithm for IoT and it would be interesting to compare its performance with other extensions of CoCoA that have been recently proposed. Furthermore, since CoAP is actively used by several IoT applications, it is important to study the fairness implications when CoAP and CoCoA++ co-exist.

Conflict of interest statement None. Declaration of confliction interests The authors declare that they had no conflicts of interest with respect to their authorship or the publication of this article. References [1] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, M. Ayyash, Internet of Things: A survey on enabling technologies, protocols, and applications, Commun. Surv. Tutor. IEEE 17 (4) (2015) 2347–2376. [2] A. Ghaffari, Congestion control mechanisms in Wireless Sensor Networks: A survey, J. Netw. Comput. Appl. 52 (2015) 101–115. [3] C. Gomez, A. Arcia-Moret, J. Crowcroft, TCP in the Internet of Things: from ostracism to prominence, IEEE Internet Comput. 22 (1) (2018) 29–41. [4] L. Atzori, A. Iera, G. Morabito, The Internet of Things: A survey, Computer Netw. 54 (15) (2010) 2787–2805. [5] L.S. Brakmo, L.L. Peterson, TCP Vegas: End to end congestion avoidance on a global Internet, IEEE J. Select. Areas Commun. 13 (8) (1995) 1465–1480. [6] A. Betzler, C. Gomez, I. Demirkol, J. Paradells, Congestion control in reliable CoAP communication, in: Proceedings of the 16th ACM International Conference on Modeling, Analysis & Simulation of Wireless and Mobile Systems, ACM, 2013, pp. 365–372. [7] J. Mo, R.J. La, V. Anantharam, J. Walrand, Analysis and comparison of TCP Reno and Vegas, in: INFOCOM’99. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, Vol. 3, IEEE, 1999, pp. 1556–1563. [8] J.S. Ahn, P.B. Danzig, Z. Liu, L. Yan, Evaluation of TCP Vegas: emulation and experiment, in: ACM SIGCOMM Computer Communication Review, vol. 25, ACM, 1995, pp. 185–195. [9] K. Kurata, G. Hasegawa, M. Murata, Fairness comparisons between TCP Reno and TCP Vegas for future deployment of TCP Vegas, in: Proceedings of INET, 2000, pp. 1–9.

1071

[10] A. Betzler, C. Gomez, I. Demirkol, M. Kovatsch, Congestion Control for CoAP cloud services, in: Emerging Technology and Factory Automation, ETFA, IEEE, 2014, pp. 1–6. [11] D.A. Hayes, G. Armitage, Revisiting TCP congestion control using delay gradients, in: International Conference on Research in Networking, Springer, 2011, pp. 328–341. [12] A. Betzler, C. Gomez, I. Demirkol, J. Paradells, CoCoA+: An advanced congestion control mechanism for CoAP, Ad Hoc Netw. 33 (2015) 126–139. [13] Get started with contiki, instant contiki and cooja, http://www.contikios.org/start.html. [14] A. Banks, R. Gupta, MQTT Version 3.1. 1. OASIS StandArd, 2014. [15] C. Bormann, A. Betzler, C. Gomez, I. Demirkol, CoAP Simple Congestion Control/Advanced, 2014, ID: draft-bormann-core-cocoa-02. [16] Z. Shelby, K. Hartke, C. Bormann, The Constrained Application Protocol (CoAP)(RFC 7252), 2014. [17] C. Borman, S. Lemay, V.S. Barboza, H. Tschofenig, A TCP and TLS Transport for Constrained Application Protocol (CoAP): Draft-Core-Coap-Tcp-Tls-05, 2017. [18] S. Kumar, M.P. Andersen, H.-S. Kim, D.E. Culler, TCPlp: System design and analysis of full-scale TCP in low-power networks, 2018, arXiv preprint arXiv:1811.02721. [19] A. Betzler, C. Gomez, I. Demirkol, J. Paradells, CoAP Congestion control for the Internet of Things, IEEE Commun. Mag. 54 (7) (2016) 154–160. [20] P. Karn, C. Partridge, Improving round-trip time estimates in reliable transport protocols, in: ACM SIGCOMM Computer Communication Review, vol. 17, ACM, 1987, pp. 2–7. [21] C. Bormann, A. Betzler, C. Gomez, I. Demirkol, CoAP simple congestion control/advanced, (Work in Progress), ID: draft-bormann-core-cocoa-03. [22] A. Pramanik, A.K. Luhach, I. Batra, U. Singh, A systematic survey on congestion mechanisms of CoAP based Internet of Things, in: Advanced Informatics for Computing Research, Springer, 2017, pp. 306–317. [23] I. Järvinen, L. Daniel, M. Kojo, Experimental evaluation of alternative congestion control algorithms for Constrained Application Protocol (CoAP), in: 2nd World Forum on Internet of Things, WF-IoT, IEEE, 2015, pp. 453–458. [24] A. Betzler, C. Gomez, I. Demirkol, Evaluation of advanced congestion control mechanisms for unreliable CoAP communications, in: Proceedings of the 12th ACM Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, & Ubiquitous Networks, ACM, 2015, pp. 63–70. [25] E. Ancillotti, R. Bruno, Comparison of CoAP and CoCoA+ congestion control mechanisms for different IoT application scenarios, in: IEEE Symposium on Computers and Communications, ISCC, IEEE, 2017, pp. 1186–1192. [26] H.M. Hasan, A.I. Ahmed, A comparative analysis for congestion mechanism in COAP and COCOA, Eng. Technol. J. 36 (8 Part (A) Engineering) (2018) 867–877. [27] S. Bolettieri, C. Vallati, G. Tanganelli, E. Mingozzi, Highlighting some shortcomings of the CoCoA+ congestion control algorithm, in: International Conference on Ad-Hoc Networks and Wireless, Springer, 2017, pp. 213–220. [28] E. Balandina, Y. Koucheryavy, A. Gurtov, Computing the retransmission timeout in CoAP, in: Internet of Things, Smart Spaces, and Next Generation Networking, Springer, 2013, pp. 352–362. [29] R. Ludwig, K. Sklower, The Eifel retransmission timer, ACM SIGCOMM Comput. Commun. Rev. 30 (3) (2000) 17–27. [30] R. Bhalerao, S.S. Subramanian, J. Pasquale, An analysis and improvement of congestion control in the CoAP Internet-of-Things protocol, in: Consumer Communications & Networking Conference (CCNC), 13th IEEE Annual, IEEE, 2016, pp. 889–894. [31] R. Hassan, A.M. Jubair, K. Azmi, A. Bakar, Adaptive congestion control mechanism in CoAP Application Protocol for Internet of Things (IoT), in: Signal Processing and Communication (ICSC), 2016 International Conference on, IEEE, 2016, pp. 121–125. [32] J.J. Lee, S.M. Chung, B. Lee, K.T. Kim, H.Y. Youn, Round trip time based adaptive congestion control with CoAP for sensor network, in: International Conference on Distributed Computing in Sensor Systems, DCOSS, IEEE, 2016, pp. 113–115. [33] E. Ancillotti, R. Bruno, C. Vallati, E. Mingozzi, Design and evaluation of a rate-based congestion control mechanism in CoAP for IoT applications, in: 2018 IEEE 19th International Symposium on a World of Wireless, Mobile and Multimedia Networks, WoWMoM, IEEE, 2018, pp. 14–15. [34] S. Bolettieri, G. Tanganelli, C. Vallati, E. Mingozzi, pcocoa: A precise congestion control algorithm for coap, Ad Hoc Netw. 80 (2018) 116–129. [35] I. Järvinen, I. Raitahila, Z. Cao, M. Kojo, Is CoAP congestion safe? in: Proceedings of the Applied Networking Research Workshop, ACM, 2018, pp. 43–49.

1072

V. Rathod, N. Jeppu, S. Sastry et al. / Future Generation Computer Systems 100 (2019) 1053–1072

[36] I. Jarvinen, I. Raitahila, Z. Cao, M. Kojo, FASOR Retransmission timeout and congestion control mechanism for CoAP, in: 2018 IEEE Global Communications Conference, GLOBECOM, IEEE, 2018, pp. 1–7. [37] C. Vallati, F. Righetti, G. Tanganelli, E. Mingozzi, G. Anastasi, ECOAP: Experimental assessment of congestion control strategies for CoAP using the WiSHFUL platform, in: IEEE International Conference on Smart Computing, SMARTCOMP, IEEE, 2018, pp. 423–428. [38] P. Ruckebusch, S. Giannoulis, D. Garlisi, P. Gallo, P. Gawowicz, A. Zubow, M. Chwalisz, E. De Poorter, I. Moerman, I. Tinnirello, et al., WiSHFUL: enabling coordination solutions for managing heterogeneous wireless networks, IEEE Commun. Mag. 55 (9) (2017) 118–125. [39] J. Mišić, M.Z. Ali, V.B. Mišić, Architecture for IoT domain with CoAP observe feature, IEEE Internet Things J. 5 (2) (2018) 1196–1205. [40] H.A. Al-Kashoash, Y. Al-Nidawi, A.H. Kemp, Congestion analysis for low power and lossy networks, in: Wireless Telecommunications Symposium, WTS, IEEE, 2016, pp. 1–6. [41] H.A. Al-Kashoash, Y. Al-Nidawi, A.H. Kemp, Congestion-aware RPL for 6LoWPAN networks, in: Wireless Telecommunications Symposium, WTS, IEEE, 2016, pp. 1–6. [42] H.-S. Kim, H. Kim, J. Paek, S. Bahk, Load balancing under heavy traffic in RPL routing protocol for low power and lossy networks, IEEE Trans. Mob. Comput. 16 (4) (2017) 964–979. [43] H.-S. Kim, J. Ko, D.E. Culler, J. Paek, Challenging the IPv6 routing protocol for low-power and lossy networks (RPL): A survey, IEEE Commun. Surv. Tutor. 19 (4) (2017) 2502–2525. [44] H.-S. Kim, J. Paek, D.E. Culler, S. Bahk, Do not lose bandwidth: Adaptive transmission power and multihop topology control, in: 13th International Conference on Distributed Computing in Sensor Systems, DCOSS, IEEE, 2017, pp. 99–108. [45] H.-S. Kim, H. Im, M.-S. Lee, J. Paek, S. Bahk, A measurement study of TCP over RPL in low-power and lossy networks, J. Commun. Netw. 17 (6) (2015) 647–655. [46] K.K. Jonassen, Implementing CAIA delay-gradient in linux (Master’s thesis), Department of Informatics, University of Oslo, Norway, 2015. [47] CoCoA++ Supporting files, 2019, https://github.com/steps-to-reproduce/ cocoapp-supporting-files.git. [48] FIT/IoT-LAB: IoT experimentation at a large scale, https://www.iot-lab.info/. [49] Zolertia Z1 mote, https://github.com/Zolertia/Resources/wiki/The-Z1-mote. [50] Telosb (Sky) mote, https://telosbsensors.wordpress.com/. [51] M.T.Y. Chan, Wide-area wireless network topology, US Patent App. 11/782, 524, Jan. 31 2008. [52] Nils Aschenbruck, Raphael Ernst, Elmar Gerhards-Padilla, Matthias Schwamborn, Bonnmotion: a mobility scenario generation and analysis tool, in: Proceedings of the 3rd International ICST Conference on Simulation Tools and Techniques, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2010, pp. 51:1–51:10, http://dx.doi.org/10.4108/ICST.SIMUTOOLS2010.8684, https:// doi.org/10.4108/ICST.SIMUTOOLS2010.8684. [53] T. Camp, J. Boleng, V. Davies, A survey of mobility models for ad hoc network research, Wirel. Commun. Mobile Comput. 2 (5) (2002) 483–502. [54] J. Ariyakhajorn, P. Wannawilai, C. Sathitwiriyawong, A comparative study of Random Waypoint and Gauss-markov mobility models in the performance evaluation of MANET, in: Communications and Information Technologies, ISCIT’06 International Symposium on, IEEE, 2006, pp. 894–899. [55] F.J. Martinez, J.-C. Cano, C.T. Calafate, P. Manzoni, Citymob: a mobility model pattern generator for VANETs, in: IEEE International Conference on Communications (ICC) Workshops, IEEE, 2008, pp. 370–374. [56] F. Bai, A. Helmy, A survey of mobility models, in: Wireless Adhoc Networks, vol. 206, University of Southern California, USA, 2004, p. 147. [57] Wireless Sensor Nodes Deployment at Grenoble, Paris and Lille Site, https: //www.iot-lab.info/deployment/. [58] MAC protocols in ContikiOS, http://anrg.usc.edu/contiki/index.php/MAC_ protocols_in_ContikiOS. [59] R. Lim, F. Ferrari, M. Zimmerling, C. Walser, P. Sommer, J. Beutel, Flocklab: a testbed for distributed, synchronized tracing and profiling of wireless embedded systems, in: Information Processing in Sensor Networks (IPSN), ACM/IEEE International Conference on, IEEE, 2013, pp. 153–165.

[60] Flocklab: A testbed for distributed, synchronized tracing and proling of wireless embedded systems, https://gitlab.ethz.ch/tec/public/flocklab/ wikis/home. [61] Cedric Adjih, Emmanuel Baccelli, Eric Fleury, Gaetan Harter, Nathalie Mitton, Thomas Noel, Roger Pissard-Gibollet, Frederic Saint-Marcel, Guillaume Schreiner, Julien Vandaele, et al., FIT IoT-LAB: A Large Scale Open Experimental IoT Testbed, in: 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT), IEEE, 2015, pp. 459–464.

Vishal J. Rathod is a Ph.D. Student in the Department of Computer Science and Engineering at National Institute of Technology Karnataka, Surathkal, India. He received his Bachelor’s degree in Computer Engineering from Gujarat University, Gujarat, India and Master’s Degree in Computer Engineering from Dharmsinh Desai University, Gujarat, India. Vishal’s areas of interests include Internet of Things, Congestion Control and Network Performance Analysis.

Natasha Jeppu is a Bachelors student in the Department of Computer Science and Engineering at National Institute of Technology Karnataka, Surathkal, India. She is a member of the Wireless Information Networking Group (WiNG). Her areas of interest include Computer Networks and Formal Verification.

Samanvita Sastry is a Bachelors student in the Department of Computer Science and Engineering at National Institute of Technology Karnataka, Surathkal, India. She is a member of the Wireless Information Networking Group (WiNG). Her areas of interest include Wireless Sensor Networks.

Shruti Singala is a Bachelors student in the Department of Computer Science and Engineering at National Institute of Technology Karnataka, Surathkal, India. She is a member of the Wireless Information Networking Group (WiNG). Her areas of interest include Internet of Things.

Mohit P. Tahiliani is an Assistant Professor of Computer Science and Engineering at National Institute of Technology Karnataka, Surathkal, India. He obtained his Ph.D in the area of Congestion Control Mechanisms for the Next Generation Internet from the Department of Computer Science and Engineering at National Institute of Technology Karnataka, Surathkal in 2013. He completed his Bachelors and Masters in Computer Science & Engineering from Visvesvaraya Technological University, Belgaum in the year 2007 and 2009 respectively. His areas of interest include Congestion Control Algorithms, Active Queue Management, Traffic Engineering, Network Performance Analysis and Network Simulations.