Routing in hybrid Delay Tolerant Networks

Computer Communications xxx (2014) xxx–xxx

Contents lists available at ScienceDirect

Computer Communications journal homepage: www.elsevier.com/locate/comcom

Routing in hybrid Delay Tolerant Networks Christoph P. Mayer, Oliver P. Waldhorst ⇑ Institute of Telematics, Karlsruhe Institute of Technology, Zirkel 2, 76131 Karlsruhe, Germany

a r t i c l e

i n f o

Article history: Available online xxxx Keywords: Delay Tolerant Network Hybrid routing Overlay Network

a b s t r a c t Delay Tolerant Networks (DTNs) have emerged as communication paradigm for providing end-to-end communication based on store-carry-forward mechanisms without the need for costly infrastructure. However, empirical studies have shown that integrating opportunistically encountered infrastructure— e.g., Internet access via WiFi—into hybrid DTNs can significantly boost routing performance. Nevertheless, extending sophisticated DTN protocols for decentralized routing towards and across the infrastructure is both complex and insufficiently understood. In this paper, we present the overlay-based Hybrid Routing System (HRS) which is—to the best of our knowledge—the first decentralized and collaborative approach for routing in hybrid DTNs that does not rely on central servers. With HRS, a large class of existing DTN protocols can benefit from opportunistic infrastructure encounters, as we show by integrating three prominent representatives of this class into HRS. In an extensive simulation study we show that (1) hybrid routing in a decentralized setting is indeed possible and can significantly boost the performance of sophisticated DTN routing protocols, (2) routing towards the infrastructure can be implemented independently for the message destination in a scalable way, and (3) communication and storage overhead can be kept low since target-oriented message forwarding across the infrastructure can avoid heavy message replication. Ó 2014 Elsevier B.V. All rights reserved.

1. Introduction The increasing pervasiveness of mobile devices enables a new class of networks that provide communication without costly infrastructure, solely based on the resources provided by the mobile devices and store-carry-forward mechanisms. Such Delay Tolerant Networks (DTN) [1–3] leverage human mobility and social behavior for providing end-to-end communication. DTNs are employed when communication infrastructure is not available, or devices are frequently disconnected such that no continuous end-to-end path may exist between sender and receiver at any time. The main challenge in DTNs is routing in face of intermittent connectivity [2]. DTN routing protocols have been proposed based on epidemic forwarding [4], social communities [5,6], or resource allocation [7]. Many current DTN routing protocols are destination-aware and forward messages based on (logical) proximity to the message’s destination device, given, e.g., by probabilities of encounters [8,9], or last encounter times [10]. Mobile devices carried by humans are often multi-homed: they can participate in a DTN and have Internet access, e.g., via WiFi or ⇑ Corresponding author. E-mail addresses: [email protected] (C.P. Mayer), [email protected], waldhorst@ tm.uka.de (O.P. Waldhorst).

cellular networks. This allows for integration of such infrastructure into a hybrid DTN. It has been shown empirically that hybrid DTNs can improve communication performance [11–13], enable communication between geographically separated DTNs, and allow for communication between devices (only) connected to the DTN and devices (only) connected to the Internet—supporting novel applications such as opportunistic computing based ‘‘Urban Sensing’’ [1], or ‘‘Not-so-instant Messaging’’ [14]. However, exploiting non-deterministic infrastructure encounters in DTN routing protocols is complex and insufficiently understood. Previous works in this area considered only very basic DTN protocols such as two-hop delivery, opportunistic flooding, and multi-copy [11–13]. Furthermore, related work did not detail on the interaction of a device connected to the infrastructure with both other devices connected at the same time or with the infrastructure itself. Often, this interaction depends on a central server [15,13], contradicting the idea of message forwarding solely based on the resources of the mobile devices. Consequently, the major challenges for using a sophisticated destination-aware protocol in a hybrid DTN is to define collaborative and decentralized mechanisms that can route messages (a) towards the infrastructure if appropriate for the message destination and (b) across the infrastructure to a device with higher proximity to the destination device, if available.

http://dx.doi.org/10.1016/j.comcom.2014.03.018 0140-3664/Ó 2014 Elsevier B.V. All rights reserved.

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

2

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

This paper tackles these challenges and presents a methodology for destination-aware routing in hybrid DTNs, called Hybrid Routing System (HRS)—naturally extending the collaborative and decentralized mechanisms of existing DTN protocols into the Internet. To the best of our knowledge, this is the first routing system that allows for scalable and transparent routing in hybrid DTNs without central servers. HRS allows to bias the DTN protocol for routing towards the infrastructure as solution to challenge (a) and offers two distributed schemes that employ key-based routing overlays for forwarding messages across the infrastructure as solution for challenge (b). To allow for deployment of the DTN protocol that works best in the target scenario, HRS provides a framework for integrating a large set of destination-aware DTN protocols. To illustrate this, we integrate three state-of-the-art destination-aware protocols—Prophet [8], MaxProp [9], and Spray&Focus [10]—into HRS as proof of concept. We conduct extensive simulation studies using real-world city maps and a model for human mobility and gain the following key findings: (1) Destination-aware routing can be effectively employed in hybrid DTNs in a decentralized setting. In fact, destination-aware DTN protocols benefit heavily from infrastructure access. For example, equipping only 30% of devices with Internet access increases the delivery probability by 169% for Prophet in the considered setting. (2) Routing towards the infrastructure can be implemented in a highly scalable way. In fact, routing decisions can be made independently of the message destination, avoiding to store proximity values for each destination currently reachable via the infrastructure. (3) Storage and communication overhead can be kept low, since heavy message replications is unnecessary due do the performance boost obtained by infrastructure availability in many cases. This paper is structured as follows: We characterize destination-aware protocols using prominent examples in Section 2. Section 3 introduces hybrid DTNs and identifies challenges for hybrid routing. In Section 4 we present the Hybrid Routing System (HRS) to solve these challenges. Section 5 provides an in-depth evaluation of the performance of HRS. Related work is discussed in Section 6. Finally, concluding remarks are given. The work presented in this paper is based on [16].

2. Destination-aware DTN protocols Delay Tolerant Networks (DTNs) build upon a store-carry-forward paradigm using local communication only: Two devices can exchange messages when they encounter, i.e., when they are located in each others transmission range using a local or personal area wireless communication technology, such as IEEE 802.11 in ad hoc mode or Bluetooth. For the network model considered in the remainder of the paper we assume that several devices move in a geographic area denoted as playground. Devices encounter when located in each others communication range, which is fixed and homogeneous for all devices. Each device has an identifier (ID) that is fixed and unique. e.g., identifiers could comprise human-readable clear text information, like email addresses of device owners, or more abstract numbers such as International Mobile Equipment Identity (IMEI) as used commonly in mobile telecommunication systems. Devices can send messages to other devices—even when not located in mutual transmission range—using the device identifier to address the destination. Messages are forwarded on device encounters using the store-carry-forward mechanism. We refer to the set of devices as D and use i 2 D to refer either to the device itself, or to its ID; depending on the context. A DTN protocol delivers a message mij from a source device i 2 D to the destination device j 2 D using local communication and storecarry-forward. In general, DTN protocols achieve this goal with

quite different mechanisms. At the one extreme some protocols use direct delivery that transmits a message mij only when i and j encounter directly. At the other extreme, with epidemic routing [4] each device k 2 D maintains a local buffer for storing messages mij , even if it is neither the source nor the destination, i.e., k R fi; jg. Devices k; l 2 D copy all messages from each others buffer upon an encounter. Obviously, direct delivery generates low overhead, while delay is high since it may take a long time until source and destination devices encounter. On the other hand, epidemic routing reduces delay, since it is more likely that the destination encounters a device carrying a copy of the message.1 However, the overhead for copying and storing messages is high. Other DTN protocols typically trade off overhead for delay by maintaining some form of routing information that helps to decide whether a particular message mij should be copied between devices k; l 2 D upon their encounter. We categorize DTN protocols with respect to the structure and utilization of routing information as unaware, self-aware, or destination-aware. Unaware protocols do not evaluate whether device l is more likely to deliver the message to device j than devices k, but rather perform (limited) flooding as in Epidemic Routing [4] or Spray&Wait [17], or replication on a permessage utility as in RAPID [7].2 Self-aware protocols evaluate the quality of devices k and l as a forwarder in general, irrespective of the message’s destination device j. SimBet [18], e.g., locally manages a device-specific rating out of social similarity and betweenness, Encounter Based Routing [19] employs a local encounter-rate that reflects how frequently the device comes into contact with other devices, or SANE [20] uses social interests of device owners. Destination-aware protocols use routing information to decide how well-suited both devices k and l are for routing the message mij towards the destination device j. Information required for this decision is gathered locally on device encounters, possibly taking indirect device encounters through a third device (transitivity) into account. Information decays over time (a device becomes less valuable) and is refreshed with further device encounters. Since the most elaborate DTN protocols can be categorized as destinationaware, e.g., [8–10], we focus on this category in the remainder of the paper. In general form, the operation of a destination-aware DTN routing protocol can be described as follows: Each device i 2 D maintains a proximity pi ðjÞ 2 ½0; 1 for all other devices j 2 D. It holds pk ðjÞ > pl ðjÞ if device k is either more likely to encounter the destination j directly or to encounter a device m with pm ðjÞ > pl ðjÞ (i.e., k will likely encounter a device m that is more likely to encounter j than l is), for i; j; k; l; m 2 D. Many proposed DTN protocols integrate the second aspect of transitivity. Consequently, consistent with [21] routing a message mij from device i to device j is a recursive process based on store-carry-forward. On an encounter of devices k and l, message mij carried by a device k is transferred to device l if l ¼ j, or

pl ðjÞ > pk ðjÞ:

ð1Þ

Note that depending on the DTN routing protocol after transferring the message mij from device k to device l it can be either deleted or kept on device k. Since this leads to either a single copy or multiple copies of the message within the DTN we refer to the mode of operation of the protocol as single-copy or multi-copy mode, respectively. Single-copy mode reduces the overhead in the networks, while multi-copy mode enhances the probability of a successful delivery.3 1 Assuming an ideal model of infinite device buffers and unlimited communication bandwidth. 2 RAPID supports multiple optimization metrics, depending on the metric employed it is categorized as destination-aware or unaware. 3 Again, assuming a model of infinite device buffers and unlimited communication bandwidth.

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

3

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

The mobility behavior of DTNs considered in this paper is not deterministic. Therefore, the proximity metrics pi ðjÞ are neither known a priori, nor fixed over time. Thus, a destination-aware protocol must define semantics of the proximity metric and implement the following tasks:

devices i and j, the values of fik and fjk are exchanged for all

1. initialize proximity, 2. refresh proximity upon device encounters, and 3. decay proximity over time.

mink1 ¼i;k2 ;...;kr ¼j cðk1 ; k2 ; . . . ; kr Þ. Note that with MaxProp proximities are only updated on device encounters, i.e., there is no decay of proximity over time (Task 3). Since we have fji 2 ½0; 1 and r < jDj, it holds 0 6 cðk1 ; . . . ; kr Þ 6 jDj and we can map MaxProp proximity to prox-

In the following we discuss how prominent destination-aware DTN protocols implement these tasks. For that purpose, we use the notation and semantics from the original papers. These may differ from the notation of proximity introduced above. e.g., proximity values may be arbitrary real numbers beyond ½0; 1, or ordering of proximity may be reverse to Eq. 1, i.e., smaller proximity indicates a better forwarder. However, we provide a mapping of proximity values maintained by each of the protocols to values pi ðjÞ used in Eq. 1. Such mapping is required for integrating a protocol into the proposed Hybrid Routing System as shown later. 2.1. Prophet Lindgren et al. presented the Probabilistic Routing Protocol using History of Encounters and Transitivity, short Prophet [8]. Prophet uses the probability P ði;jÞ for devices i and j to meet as proximity metric. To initialize proximity, Prophet sets Pði;jÞ ¼ 0 if devices i and j have never met (Task 1). To refresh proximity on a device encounter (Task 2), on the first encounter of these devices Prophet sets P ði;jÞ ¼ pinit 2 ð0; 1Þ. To update the proximity metric on future encounters of devices i and j, Prophet sets P ði;jÞ ¼ Pði;jÞold þ ð1  P ði;jÞold Þ  pinit . To account for transitivity, Prophet additionally sets P ði;kÞ ¼ P ði;kÞold þ ð1  Pði;kÞold Þ  Pði;jÞ  P ði;kÞ  b, using a scaling factor b 2 ½0; 1, for all devices k 2 D n fi; jg on every encounter of devices i and j. To decay proximity over time (Task 3), Prophet sets P ði;jÞ ¼ Pði;jÞold  ct , with aging constant c 2 ½0; 1Þ, if pi ðjÞ has neither been updated due to a direct encounter nor due to transitivity for t > 0 time units. This implies that Prophet proximity values are (virtually) updated continuously over time. However, it is sufficient to perform updates only when messages or proximity values are exchanged on an encounter. Furthermore, these updates can be computed locally by each device, without any intervention from other devices reporting these values. The authors propose in [8] to use pinit ¼ 0:7; c ¼ 0:98, and b ¼ 0:25. With the semantics of proximity defined by Prophet we have P ði;jÞ 2 ½0; 1 and Pðl;jÞ > P ðk;jÞ iff a message mi;j is forwarded from k to l on an encounter. Thus, we can directly map Prophet proximity pi ðjÞ used in Eq. 1, i.e., pi ðjÞ ¼ Pði;jÞ . 2.2. MaxProp Burgess et al. presented MaxProp in [9]. MaxProp combines unaware and destination-aware mechanisms, i.e., it replicates messages that have not traveled far—identified by a hop counter—on device encounters and uses a proximity measure only to break ties. To this end, it derives the proximity between devices i and j from a cost metric cðk1 ; k2 ; . . . ; kr Þ. This cost metric is determined by the probability of a sequence of r pairwise encounters between devices kp and kpþ1 , with 1 6 p < r; kp 2 D; k1 ¼ i; kr ¼ j, that may happen in the future. To compute the cost metric c, each device i 2 D locally calculates encounter probabilities fij for every other device j 2 D. 1 if devices i and j; i – j, have never These are initialized to fij ¼ jDj1

met (Task 1). To refresh proximity, meeting probabilities are updated by incremental averaging, i.e., on an encounter of devices i and j; fij is incremented by 1 and all values fik ; k – i, are re-normalized in order to sum up to 1 (Task 2). On encounter of

k 2 D. To incorporate transitivity, the devices exchange fkl for k; l 2 D that they have gathered in previous encounters. Based on these meeting probabilities, each device can  P  ksþ1 compute the proximity metric cðk1 ; . . . ; kr Þ ¼ r1 and s¼1 1  fks

imity pi ðjÞ used in Eq. 1 by pi ðjÞ ¼ 1 

mink

1 ¼i;k2 ;...;kr ¼j

jDj

cðk1 ;...;kr Þ

.

2.3. Spray&Focus Spyropoulos et al. presented Spray&Focus [10], a protocol that combines an unaware spray phase with a destination-aware focus phase. A newly generated message mij is associated with L forwarding tokens. For that message, a device i operates in spray phase if it carries the message and L > 1 forwarding tokens. On an encounter with a device j that does not carry the message, i copies the message to j. Furthermore, device i hands over half of the forwarding tokens to device j, i.e., the number of forwarding tokens at the devices is set to bL=2c and dL=2e for i and j, respectively. When the number of forwarding tokens at a device reaches L ¼ 1, the device switches to the destination-aware focus phase. Here, proximity is given by the time si ðjÞ since a device i has encountered the destination j either directly or indirectly taking transitivity into account. To initialize proximity, Spray&Focus sets si ðjÞ ¼ 1 (Task 1). To refresh proximity on a device encounter, it sets si ðjÞ ¼ 0. Furthermore, it sets si ðkÞ ¼ sj ðkÞ if sj ðkÞ < si ðkÞ for all k – i; j to account for transitivity. To decay proximity over time (Task 3), si ðkÞ is incremented by one on every time unit. As with Prophet, proximity updates are required only when proximity values are used and can be computed by every device locally, without external input. Obviously, it holds si ðjÞ P 0. Thus, we define a maximum time horizon d > 0 and map Spray&Focus proximity to proximity pi ðjÞ n o used in Eq. 1 by pi ðjÞ ¼ 1  min sidðjÞ ; 1 . 3. Hybrid DTNs The DTN protocols described so far rely on encounters and local communication between mobile devices to forward/replicate messages. In particular, forwarding does neither require Internet access, nor depend on any other infrastructure such as access points or base stations. However, if such infrastructure is available, it would be negligent not to use it, as taken into account in several fundamental works [11,13,12]. Such work in general assumes the existence of several access points, to which the mobile devices can connect to when moving into their coverage. Such scenarios, that we refer to as hybrid DTNs, have two benefits:  First, infrastructure enhances the probability that two devices encounter directly, because a device that is connected to the Internet via an access point can be considered to be (virtually) connected to all other devices currently connected to the Internet [11,12]. This may help to find shortcuts to devices that are located many DTN hops away or even in a different, weakly connected DTN cluster.  Second, encounters can be decoupled in time, i.e., a device that connects to the infrastructure can leave a message—for example at a central server with a well-known address as proposed in [11,13]—that can be picked up by a second device connecting later when the first device has already disconnected.

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

4

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

In this paper we consider hybrid DTNs based on a network model quite similar to that used in [11,13,12]. In addition to access points that enable Internet access when a device enters their coverage, we also consider devices equipped with permanent Internet access, e.g., using a 3G cellular connection. These can be considered as some type of moving access points for other DTN nodes. Fig. 1 shows an example for a hybrid DTN. As with classical DTNs, devices from a set D move in a plane and can exchange messages when located in mutual communication range (e.g., f ; g in Fig. 1). To incorporate the ‘‘hybrid’’ aspect, we partition the set of devices D depending on their capability for accessing the infrastructure that provides communication to the Internet:  permanent set Dp (devices having permanent infrastructure access, e.g., devices d; g in Fig. 1),  temporary set Dt (devices having temporary infrastructure access in the coverage of an access point, e.g., devices c; e; j), and  not available set Dn (devices not capable of accessing the infrastructure, e.g., devices a; b; f ; h; l; k). The goal of our work is to gain an understanding of and provide mechanisms for the interaction of sophisticated destination-aware DTN protocols with the infrastructure in hybrid DTNs. As example consider the most complex case of such interaction: sending a message between devices without infrastructure access, e.g., message mak from a to k (a; k 2 Dn ), in three steps: (a) Routing in the DTN towards the infrastructure: Forwarding mak from a by DTN routing to a device that is likely to have/ encounter infrastructure access (e.g., over b to c). (b) Routing across the infrastructure: Forwarding mak from c to another device currently connected to the infrastructure (e.g., g) with a higher proximity to the destination k. (c) Routing in the DTN to the destination: Forwarding mak by DTN routing over j to the destination k. Obviously, Step (c) can be accomplished by a destination-aware DTN protocol without any modifications. Thus, we mainly focus on Steps (a) and (b) in the remainder of this work. Step (b) is most complex and raises the question how a device i that connects to the infrastructure interacts with both the infrastructure itself, and other devices currently connected. Device i must 1. discover other devices currently connected to the infrastructure, probably at a different access point,

2. forward messages stored locally to devices with higher proximity, deposit messages with no proper forwarding available, and pick up messages it could forward itself, as well as 3. update its proximity information pi ðjÞ and trigger updates of pj ðiÞ at other devices j. While demonstrating that such interaction may help to improve DTN performance, related work does not provide (all) details of the aspects (1) to (3). More specific, aspect (1) is not discussed in [11,12]. In [13] a central well-known server is used for storing and picking up messages, making aspect (1) unnecessary. For aspect (2), related work considers direct delivery (involving a central server) [11], two-hop delivery [13], or unaware protocols such as opportunistic flooding and multi-copy [12], as well as RAPID [13]. Here, destination-aware DTN-protocols require more sophisticated mechanisms. Aspect (3) is not considered in [11–13] at all, since none of the considered DTN protocols requires a proximity metric maintained at each mobile device. 4. The Hybrid Routing System We would like to point out that approaches considering a central server [11,13] to some extend contradict the idea of a DTN, in which message delivery is performed using only resources of the mobile devices, without the support of any infrastructure. Thus, we explore fully decentralized approaches—solely based on the resources provided by the mobile devices—for all three aspects raised above. Since mobile devices typically have limited resources, such approaches must reduce communication overhead and memory consumption as far as possible. In particular, they must scale with the number of devices participating in the system. This directly affects discovery, message forwarding and storage, and it has implicit impact on the updating mechanism for proximity information: Since millions of DTN devices may be connected to the Internet at the same time, we can neither afford to update, nor to store proximity information for all these devices at a single DTN device. Thus, we must provide ways to cope with incomplete proximity information in both Steps (a) and (b) of the forwarding process. To solve Step (b), we present mechanisms for distributed discovery, storage, and message forwarding in the infrastructure consistently with the proximity information provided by a destination-aware DTN protocol in Sections 4.1 and 4.2. The problem of coping with incomplete proximity information in Step (a) is solved by biasing the DTN protocol for routing towards the infrastructure, and will be discussed in Section 4.3. Note that related work [11,13,12] does not give details on the scalable implementation of a centralized infrastructure component for DTNs, which has to cope with the same fundamental problems as a decentralized one. To this end, many of the mechanisms discussed throughout this paper can also be used to make a centralized solution (more) scalable. 4.1. Decentralized forwarding and storage using key-based routing overlays

Fig. 1. Hybrid network model.

The Hybrid Routing System (HRS) presented in this paper solves the challenges stated for Step (b) in Section 3: For providing distributed discovery, forwarding, and storage capabilities HRS uses a structured overlay network maintained by the DTN devices that are currently connected to the Internet. Structured overlay networks (or for short overlays) are elaborate tools that can provide both distributed storage and targeted forwarding between the devices based on devices identifiers. Many structured overlays have been proposed in literature, e.g., [22,23]. Although in general they provide services with slightly different

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

semantics, it has been shown that all of them can be used to implement key-based routing (KBR) as fundamental abstraction [24]. We refer to a structured overlay that implements the KBR abstraction as KBR overlay. Using the notation from [24], we refer to a node n 2 N as an instance of a participant (in our case, a device participating in the DTN with current infrastructure access) in the structured overlay. Here, N denotes the set of all nodes that are currently part of the KBR overlay, with a total number of N ¼ jN j nodes. Note that a single participant/device can instantiate more than one node in the KBR overlay. Each node is identified by a uniform random node identifier idðnÞ 2 I from a large identifier space I . We use n 2 N to refer to both the node n and its identifier idðnÞ, if unambiguous. Well-known KBR overlay protocols are Chord [22] and Pastry [23], which both use circular identifier spaces of size 2160 and 2128 , respectively. The purpose of KBR overlays is to handle application-specific data objects, which are assigned IDs from the same identifier space as the nodes. Such object IDs are denoted as keys. A KBR overlay maps a key k to a unique root node rðkÞ. In general, the mapping of the key to its root depends on a distance metric d : I  I ! R, which is defined on the identifier space. For example, in Chord, the root is defined as the closest ID of a node clockwise in the ring. In Pastry, the root is determined as the node with the smallest absolute value of the difference between the key and the node ID. The functionality provided by the key-based routing abstraction is to deliver a message addressed to a key k to the root rðkÞ of that key. The delivery process is controlled by routing tables maintained by each node that determine the next hop node for a particular key. In general, a message traverses multiple nodes before reaching the root. Typical structured overlays with N nodes deliver messages in Oðlog NÞ hops using routing tables of the same size. All operations for overlay maintenance generate Oðlog NÞ messages, implying that the overlay scales with the number of nodes. Note that a message addressed to a key k that has no exact match to any node ID in the overlay is delivered to the closest (root) node rðkÞ determined by the distance metric. This behavior is called closest match property. HRS uses a KBR overlay to implement forwarding and storage of messages. It does not depend on a particular type of KBR overlay, but can employ any structured overlay that implements the keybased routing abstraction together with the closest match property by dealing with the distance metric accordingly. For ease of exposition, we focus the description in the remainder of this paper on how to use the Pastry KBR overlay [23]. To apply KBR for both forwarding and storage of DTN messages, devices in Dp as well as the subset of devices in Dt with current infrastructure access construct the overlay network. A HRS device i with current infrastructure access joins the overlay, using a node identifier derived from its DTN identifier i. To match the requirement of a uniform distribution of node identifiers in the identifier space I , e.g., a cryptographic hash function such as SHA1 can be applied on the DTN identifiers. For ease of exposition, we additionally use i to refer to the node identifier of device i 2 D, as long as a confusion with the device identifier is not possible. A node connected to the infrastructure can use the KBR overlay to route messages to any other node through the KBR overlay using key-based routing, given that the destination node is connected to the infrastructure, too. Otherwise, HRS either forwards the message to another node that has high proximity for the destination device, or selects a node for storing the message until such a node becomes available. This requires a mapping of the proximity metric of the DTN routing protocol to the distance metric used in the KBR overlay. In the following section we describe how this mapping is provided.

5

4.2. Mapping DTN proximity to KBR distance In KBR routing, nodes in the overlay arrange with respect to their identifiers and the distance metric into a structure, e.g., a ring in the case of Pastry [23]. We efficiently exploit this structure for announcing and finding DTN devices with high proximity and current infrastructure connectivity—called proxies—by two generic decentralized overlay schemes, vHRS and iHRS. vHRS exploits the fact that one device can place additional nodes in the overlay, denote as virtual nodes. As key idea, a device i places a virtual node v ij in the overlay depending on the proximity pi ðjÞ for device j: The higher the proximity pi ðjÞ, the closer the virtual node v ij is placed to the overlay position of j. The closest match property ensures that a message forwarded towards the key j reaches either j, or the device i that has placed the virtual node v ij with smallest distance. Details on the positioning are given in Section 4.2.1. iHRS exploits the closest match property to establish the mapping between proximity and distance by indirection, inspired by approaches like i3 [25]. In simple words, for each DTN device not connected to the infrastructure we pick a unique one of the devices currently connected. This device acts as a proxy admin and maintains a list of all current proxies of the destination and their proximity to the destination device. More precisely, the device that has currently the smallest identifier distance to the destination is choosen as proxy admin. Whenever a message is sent to the destination via KBR, it reaches the proxy admin, which forwards the message to one or multiple proxies with currently highest proximity. The main operations that are defined by iHRS and vHRS—joining and leaving the overlay, forwarding and storing a message, and updating proximity information both on a device encounter and periodically—are described in the following Sections 4.2.1 and 4.2.2, respectively. 4.2.1. vHRS: hybrid routing based on virtual nodes Joining the overlay: As described above, vHRS is based on inserting additional virtual nodes into the overlay, exemplary shown in Fig. 2. Virtual nodes are used to represent proxies for devices that are by themselves (currently or generally) not capable of accessing the infrastructure. Key to the vHRS approach is the positioning of virtual nodes in the overlay. Recall that for a DTN device i 2 D we can assume that a destination-aware protocol maintains device-specific proximity values pi ðjÞ for other devices j 2 D, with pi ðjÞ 2 ½0; 1 for i – j, and pi ðiÞ ¼ 1, as described in Section 2, Upon infrastructure access, device i first joins the HRS overlay using i as KBR identifier. Furthermore, i inserts virtual nodes v ij for devices j 2 D n Dp with pi ðjÞ > 0 for which it may become a proxy (Step (1) in Fig. 2). Obviously, it is not necessary to insert virtual nodes for devices j 2 Dp with permanent infrastructure access,

Fig. 2. DTN contact graph and vHRS overlay.

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

6

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

since they will always be members of the overlay themselves. Information about membership to Dfp;t;ng is exchanged on device encounters and propagated with the awareness values. Furthermore, if pi ðjÞ ¼ 0, no messages for device j will be forwarded to device i and a virtual node is unnecessary, too. To reduce the number of virtual nodes maintained by device i, we use a threshold pjoin 2 ½0; 1 and insert virtual nodes only for devices j 2 V i from i’s virtual set given by:

V i ¼ fj : pi ðjÞ > pjoin ^ j R Dp g

ð2Þ

We can further limit the size jV i j 6 v max by selecting the vmax devices j with the highest proximity pi ðjÞ. A virtual node v ij for device j inserted by the proxy i is not inserted with j as overlay identifier, but with a virtual node identii fier idj ¼ f ðpi ðjÞ; jÞ computed as a function of both pi ðjÞ and j. The i function f is chosen such that limpi ðjÞ!1 idj ¼ j. i.e., the higher the proximity pi ðjÞ of device i to device j, the closer vHRS places the virtual node to the position where the real node in the overlay would reside. Thus, exploiting the closest-match behavior of KBR protocols, a message sent to key j is delivered either to device j itself— if j is currently part of the overlay—or to the device i in the overlay with best proximity pi ðjÞ to j and, thus, with closest placement of a virtual node v ij to j in the overlay. i For positioning a virtual node v ij at idj ¼ f ðpi ðjÞ; idðjÞÞ, vHRS divides the 160 bit overlay identifier key space into 2M segments of equal size by using:

1  pi ðjÞ jI j f ðpi ðjÞ; jÞ ¼ j   1  pjoin 2  M

! ð3Þ

Here, jI j denotes the cardinality of the identifier space I and M denotes the (maximum) number of nodes inserted into the KBR overlay. M can be a rough (design-time or runtime) approximation of the expected number of nodes. The division into 2M segments results in guard spaces on the identifier key space between the segments used for virtual node placement. For Pastry as KBR protocol, placement can be randomly left or right from the actual node posij

tion [23]. Thus, a virtual node is placed either ‘‘left’’ (idi < j) or j ‘‘right’’ (idi > j) of the actual node identifier j 1pi ðjÞ distance 1pjoin . Here, the denominator 1  pjoin 4

(cf. ‘‘2’’ in Eq. 3) with guarantees better uti-

lization of the overlay identifier space. Sending and storing messages: For sending a message mmj across the infrastructure with vHRS, a connected device l uses KBR routing with the KBR key derived from the DTN identifier of the destination j (Step (2) in Fig. 2). Subsequently, mmj is received by the proxy i that placed virtual node v ij closest to j (Step (3) in Fig. 2). l includes its proximity pl ðjÞ in the message, so the receiver i can check if pi ðjÞ > pl ðjÞ holds. This way, the condition given by Eq. 1 is even met if the identifier of virtual nodes does not reflect the current proximity due to pending refresh and decay operations. A device i that receives a message mmj can start forwarding it in the DTN if it encounters a device with higher proximity to the destination j, implicitly starting Step (c). Until such encounter happens, the message is stored by i and can be handed over when a device k with pk ðjÞ > pi ðjÞ places a virtual node v kj in the overlay. Since v kj is the new root of key j, handing over the message from i to k is implicitly triggered by the update mechanisms of the KBR overlay. This also implies that the message will be picked up by the destination node j, as soon as j joins the overlay, providing time-decoupling similar to what is proposed in related work [11,13,12], but in a decentralized setting. Note that it is also possible

4 For example, expecting 50,000 nodes, threshold pjoin ¼ 0:7, and pi ðjÞ ¼ 0:99999 j the resulting distance of idi from j in the identifier space is still > 231 due to the large identifier space of 2160 in the case of Pastry.

to implement multi-copy mode in the infrastructure by sending the message along the overlay to nodes close to the root node. Updating proximity: Recall that with destination-aware protocols proximity may either be refreshed on device encounters, or decay over time. With vHRS, a change of pi ðjÞ implies a change of i idj ¼ f ðpi ðjÞ; jÞ. Since updating node IDs is not an operation supported by KBR overlays, this requires removing the virtual node and re-inserting it with the new ID. This may cause Oðlog NÞ control messages in the overlay. Leaving the overlay: Since a node does not store any state beyond the state required for maintaining the KBR overlay, it may leave the overlay without further actions, e.g., when moving out of the coverage of an access point. If the message should be stored in the overlay, it can be replicated to additional proxies in descending order of proximity. These can be discovered by following the ring structure of the KBR overlay.

4.2.2. iHRS: hybrid routing based on Indirection Joining the overlay: iHRS decouples virtual node placement from awareness-values through indirection by means of so-called proxy admins, as shown in Fig. 3. Whereas in vHRS devices announce themselves as proxies by virtual nodes, iHRS employs proxy admins that manage proxy information on behalf of proxies, implemented by an indirection step in the overlay. We define a registration as triple fi; j; pi ðjÞg as follows: Device i 2 D registers as proxy for device j 2 D with a proximity pi ðjÞ. Upon joining the iHRS overlay, i sends a registration for each node j 2 V i for its virtual-set V i , defined in Eq. 2, to destination j using keybased routing (Step (1) in Fig. 3). Due to the closest match property, a registration reaches the root k ¼ rðjÞ of key j, i.e., the node with an identifier closest to j. Since all KBR messages to key j will reach k, KBR’s closest match property implicitly elects k as proxy admin, who will store i’s proxy registrations for j (Steps (2) and (3) in Fig. 3). Sending and storing a message: a message mmj from m to j sent over the overlay by l is addressed to key j via KBR (Step (4) in Fig. 3). It will reach either j directly or—if j is currently not part of the overlay—the proxy admin k. k can now forward the message in the overlay based on proxy registrations it has received for j (Step (5) in Fig. 3). If multiple devices have registered, selection can be performed based on the highest proximity. Note that a proxy admin k keeps multiple registrations instead of only the one with the highest proximity value. On the one hand, this can improve robustness, since other proxies are available in the case that the proxy with highest proximity suddenly looses infrastructure connectivity and, subsequently, leaves the overlay ungracefully. On the other hand, this also enables multi-copy mode in the infrastructure, since a message can be forwarded to multiple proxies. If no (or insufficient) registrations with appropriate proximity are available, the proxy admin k stores the message—again providing

Fig. 3. DTN contact graph and iHRS overlay.

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

7

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

decentralized time-decoupling similar to the centralized approaches in related work [11,13,12]. If a new registration with an appropriate proximity pi ðjÞ reaches k, the message can be delivered to the registering proxy i. Updating proximity: As a benefit of using indirection, iHRS can handle changes of proximity values that happen without encounter by the proxy admins without any interaction with the proxy, as described in Section 2. In case that proximity values need to be refreshed on a device encounter—eventually causing a change in the virtual set—the proxy sends an update to the proxy admin, which in turn can update its registration table. Leaving the overlay: If a device leaves the overlay, registration tables as well as stored messages need to be transferred to a new proxy admin. In case of a graceful leave, that can be handled by the mechanisms provided by the KBR overlay. The same holds for ungraceful leaves, if appropriate mechanisms, e.g., redundancy schemes, are provided by the KBR overlay. Otherwise, we rely on the soft-state behavior that iHRS employs—similar to i3 [25]— where awareness-values are updated due to encounters in the DTN. 4.2.3. Comparing vHRS and iHRS As advantage of vHRS state is only managed locally, i.e., each node stores only KBR state of its (real and virtual) nodes. In contrast, with iHRS information about proxies must be distributed and maintained by the proxy admin. Thus, a proxy in vHRS controls its load by the number of its virtual nodes and is not affected by other proxies, while the load of a proxy admin in iHRS is determined by the number of registrations sent by the proxies. As a consequence, vHRS allows more fine-grained load control for an individual device. Furthermore, vHRS allows for shorter overlay paths than iHRS, as messages sent to a virtual node are directly received by the proxy. On the other hand, vHRS imposes additional load, as virtual nodes are inserted and removed from the overlay due to new node identifier calculations to reflect changing proximity (cf. Eq. 3). Thus, vHRS is tailored for the case of proximity values pi ðjÞ that are not subject to heavy fluctuation. This is the case if the DTN protocol is based on long-term contacts and proximity-values that are reaching steady-state [26]. In case of highly dynamic proximity values, iHRS should be preferred to vHRS. 4.3. Routing towards the Infrastructure The mechanisms described in Section 4.2 handle routing task (b) as defined in Section 3. For full integration of an existing DTN protocol with HRS, we specify how to route in the DTN towards the infrastructure, i.e., Step (a). Recall that for reasons of scalability we cannot propagate the proximity to every device connected to the infrastructure into the DTN. For that reason, we provide a heuristic that enables to control the priority for routing towards the infrastructure. Instead of maintaining a proximity value for every device that is reachable via the infrastructure, we maintain a single proximity value for a virtual device I representing the infrastructure. Destination-aware protocols allow easy incorporation of proximity to such device. Each device i 2 D locally manages proximity pi ðIÞ to the virtual device I, reflecting the utility of device i for routing a message towards the infrastructure. We define pi ðIÞ ¼ 0 for i 2 Dn ; pi ðIÞ ¼ 1 for i 2 Dp , and pi ðIÞ 2 ½0; 1 for i 2 Dt . Whereas pi ðIÞ is fixed for i 2 Dn [ Dp , devices in Dt need to manage pi ðIÞ autonomously using the same mechanisms as for maintaining proximity values pi ðjÞ for ‘‘real’’ device j 2 D, as described in Section 2. Using a single virtual device entry for all devices reachable via the infrastructure raises the question how to decide whether to route a message mij directly to device j (i.e., using destination

address j), or route it towards the infrastructure first (i.e., using destination address I instead). To answer this question, HRS uses different approaches in Steps (a) and (c) defined in Section 3: Building upon the assumption that it is sufficient to route across the infrastructure once, in Step (a) (routing towards the infrastructure) HRS uses a configuration parameter a to trade off between routing towards the infrastructure and routing towards the destination using the DTN. In Step (c) (routing towards the destination) HRS never routes towards the infrastructure. More precisely, to take probability of infrastructure access into account when forwarding in the DTN in Step (a), we use the relative normalized values for device-proximity and infrastructureproximity of devices k; l 2 D, given by p0k;l ðjÞ ¼ p p0k;l ðIÞ ¼

pk ðIÞ . pk ðIÞþpl ðIÞ

pk ðjÞ k ðjÞþpl ðjÞ

, and

Normalized destination-awareness p0k;l ðjÞ is com-

puted only for real devices, i.e., without considering the virtual device I. Extending Eq. 1 from Section 2, a message to mij is forwarded from device k to device l with consideration of the weight parameter a 2 ½0; 1 iff:









a  p0l;k ðIÞ þ ð1  aÞ  p0l;k ðdj Þ > a  p0k;l ðIÞ þ ð1  aÞ  p0k;l ðdj Þ

ð4Þ

This implies that messages are forwarded towards devices that are more likely to encounter either the destination or a device with infrastructure access, depending on a. For selecting a good a value we show in Section 5.2 that preferring to route towards infrastructure first (a ! 1) increases message delivery ratio, even with a small number of infrastructure-enabled devices. When a message has been successfully delivered across the infrastructure, HRS ensures that the receiving device in the overlay has a higher proximity to the destination than the sender. Subsequently, the message is marked. Marked messages are routed by normal DTN routing in Step (c) using Eq. 1 without taking proximity to the infrastructure pi ðIÞ into account. 5. Evaluation A major question addressed in this paper that goes beyond related work [11,13,12] is how much a destination-aware DTN routing protocol can benefit from the availability of infrastructure in a distributed setting. To answer this questions, we look at the performance of a hybrid DTN driven by HRS with varying fractions of devices capable of accessing the infrastructure. These fractions range from a base line given by ‘‘pure’’ DTN scenarios without any infrastructure access to a trivial upper bound where all devices are connected to the infrastructure and can communicate directly. In contrast to related work, we focus manly on devices i 2 Dp with permanent infrastructure access and only briefly discuss the impact devices j 2 Dt with temporary infrastructure access enabled by access points in Sections 5.2 and 5.3. Since biasing the DTN protocol for routing towards the infrastructure by setting the parameter a appropriately might be crucial for the performance of hybrid DTNs, we will explore this aspect first in Section 5.2 for different fractions of infrastructure-capable devices, message destination models, and message time-outs. After that, we explore the performance gains of a destination-aware DTN protocol in a hybrid DTN in Sections 5.3. Obviously, these gains come at a cost in terms of storage- and communication overhead, as shown in Section 5.4. The impact of advanced single- and multicopy modes is discussed in Section 5.4. 5.1. Simulation setup For evaluation we use the ONE [27] DTN simulator extended with access points (used by devices from Dt ) and permanent infrastructure connectivity mechanisms (used by devices from Dp ).

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

8

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

Devices are randomly assigned a class Dfp;t;ng based on probabilities Pfp;t;ng 2 ½0; 1. We place devices on a playground given by the city street map of Karlsruhe, Germany that we converted5 from OpenStreetMap6 data. Devices follow a working day model similar to [28]. Devices move at walking speed for short distances, and accelerate to driving speed for distances of >2 km, which happens quite rarely due to the playground size. Devices randomly select one home location, one workplace, and between 3–5 leisure places that are shared with other devices. This results in a periodic day-by-day behavior and exhibits a power-law inter-contact-time distribution with exponential decay as found in many real world mobility traces. For details on the mobility model we refer to [28]. Every device i 2 D generates messages mij to destinations j 2 D n fig, following a message destination model: 1. Uniform: j is chosen uniformly at random. 2. Proximity-based: j is chosen randomly with probability PðjÞ  pi ðjÞ for the proximity pi ðjÞ maintained by the DTN protocol. 3. Distance-based: j is chosen at random with probability PðjÞ  1=Dði; jÞ, for the Euclidian distance Dði; jÞ of devices i; j on the playground. Intervals between sending two messages are chosen uniformly at random from 10–20 min, which is not motivated by a specific application, but sufficient to test routing performance periodically while allowing devices to move sufficiently far between successive messages. We consider a timeout value ttl 2 f1 h; 3 h; 6 hg that is fixed for all messages in a simulation run and discard messages that not delivered within the timeout. We use the DTN protocol implementations for Prophet [8], MaxProp [9], and Spray&Focus [10] provided by ONE. Message queue size is set to infinite to prevent message replacement effects that distort the hybrid routing performance we want to evaluate. In Prophet we either forward or copy messages to devices with higher proximity for single- and multi-copy mode, respectively. In MaxProp, forwarding is not based on hop count but on proximity only for single-copy, while the original protocol is used for multicopy mode. For Spray&Focus we set the initial number of forwarding tokens to L ¼ 1 and L ¼ 10 for single- and multi-copy mode, respectively. For forwarding messages across the infrastructure, we simulate a generic KBR overlay that supports the functionality of iHRS7 described in Section 4.2.2, but omit maintenance overhead of the overlay for simplicity. We set the HRS parameter pjoin ¼ 0:7 and v max ¼ 10, since these induce low overhead in practical applications. However, we would like to point out that we analyzed different values in experiments not shown here with similar results. The overlay does not perform KBR-replication of the messages to different nodes for redundancy, regardless whether the DTN protocol operates in single- or multi-copy mode. We simulate a full day and record statistics during 9 am to 9 pm. We conducted a minimum of 30 independent replicates of each experiment and present average values and 95% confidence intervals. Some simulation parameters are changed in different experiments, while others are fixed to the defaults shown in Table 1. The default values may seem arbitrary at this point; we justify our choice during the description of the experiments.

5

http://www.tm.kit.edu/mayer/osm2wkt. http://www.openstreetmap.org. 7 Since the working day model employed here does not result in stable proximity values, we do not consider vHRS in the experiments, since it will achieve lower performance and higher overhead, cmp. [16]. 6

Table 1 Common parameters used in simulations. Category

Parameter

DTN

70 Devices Communication range 10 m P p ¼ 0:2; P n ¼ 0:8; P t ¼ 0 Playground 2  2 km, street map of Karlsruhe (Ger) Walking speed 1–3 ms, driving speed 9–28 ms WiFi placement random

Message

Generation per device every 10–20 min ttl ¼ 1 h Destination models uniform, proximity, distance Message queue infinite Performance measure 9 am – 9 pm

HRS

pjoin ¼ 0:7 v max ¼ 10

5.2. Routing towards the Infrastructure We first illustrate the impact of biasing the DTN protocol to route towards infrastructure by a in Eq. 4. Fig. 4(a) and (d) plot delivery ratio, i.e., the ratio of successfully delivered messages to sent messages, for Prophet as a function of the fraction of infraPp with P t ¼ 0. Here, we use differstructure-enabled devices x ¼ Pp þP n ent values of a, different timeouts ttl, the uniform destination model, and single- and multi-copy mode, respectively. The first finding from Fig. 4(a) is that infrastructure availability boosts performance of the DTN protocol. This is expected and consistent with related work [11,13,12], although we consider a much more detailed setup with a destination-aware DTN protocol and HRS for message forwarding towards and across the infrastructure. Beyond that, the figure shows that it is not required to route towards the infrastructure explicitly: Prophet benefits from infrastructure when setting a ¼ 0, i.e., messages are delivered to infrastructure-enabled devices only by chance. Nevertheless, for ttl ¼ 1 h, the impact of a is clearly visible for a wide range of x values. e.g., for x ¼ 0:2, choosing a ¼ 1 can boost delivery ration from 0.303 to 0.387, i.e., by 27.7% compared to a ¼ 0. This trend holds for other ttl values, even though for a smaller range of x values. Fig. 4(d) confirms this for multi-copy mode, although the impact of a is less obvious since multiple copies of a message are more likely to reach at least one infrastructure-capable device without being routed there explicitly. We conclude from Fig. 4(a) and (d) that routing towards the infrastructure first is always a good choice for a uniform distribution of message destinations and use a ¼ 1 as default value in the following experiments, if not stated otherwise. Since the impact of iHRS is most visible for P p ¼ 0:2 and ttl ¼ 1 h, we additionally use these values as default in many of the following experiments. One may argue that routing towards the infrastructure is a good choice for far destinations, and, thus, is favored by uniform message destinations. To illustrate that this is not the case, Fig. 4(b) and (e) illustrate the impact of proximity- and distance-based destinations, respectively, for both single- and multi-copy mode. Consistent with Fig. 4(a) and (d) we find that routing towards the infrastructure using a ¼ 1 is the best choice for both message destination models. For proximity-based destinations the parameter a has even higher impact than for uniform destinations. This may lead to the impression that the series of encounters that has resulted in a high proximity value is quite unlikely to occur again within a short ttl, so that the proximity computed by the routing protocol is not better than random guessing and a shortcut through the infrastructure is always beneficial. However, recall that also the shortcut is determined by the routing protocol and works very effective even with few infrastructure enabled devices as shown in the experiments.

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

9

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

ttl 6h

Delivery ratio

ttl 3h

0.6

ttl 1h 0.4 α=1.0 α=0.5 α=0.0

0.2 0 0

0.2

0.4

0.6

1

0.8

0.8

0.6

0.2 0 0.8

0

1

ttl 6h ttl 3h

Delivery ratio

Delivery ratio

0.8

ttl 1h

0.6 0.4

α=1.0 α=0.5 α=0.0

0.2 0 0

0.2

0.4

0.6

0.2

Fraction of infrastructure-enabled devices

0.4 0.2 0

0.8

0

1

0.2

0.4

0.8

0.8

0.6 multi, α=1.0 multi, α=0.5 multi, α=0.0 single, α=1.0 single, α=0.5 single, α=0.0

0.4

0

0.2

0.4

0.6

0.6

0.8

1

α

1

0 1

0.6

0.6

1

0.2

0.8

0.4

Prophet MaxProp SprayFocus

Fraction of infrastructure-enabled devices

Fraction of infrastructure-enabled devices

1

multi, α=1.0 multi, α=0.5 multi, α=0.0 single, α=1.0 single, α=0.5 single, α=0.0

0.4

Delivery ratio

Delivery ratio

0.8

1

Delivery ratio

1

0.6 0.4 Prophet MaxProp SprayFocus

0.2 0 0.8

1

Fraction of infrastructure-enabled devices

0

0.2

0.4

0.6

0.8

1

α

Fig. 4. Impact of a for routing towards the infrastructure.

Furthermore, for this destination model single-copy mode with a ¼ 1 is equivalent to multi-copy mode with a ¼ 0 in terms of delivery ratio, implying that for such configurations extensive message replication can be avoided. For distance-based destinations, the impact of a is not that obvious. Nevertheless, Fig. 4(b) and (e) indicate that a ¼ 1 is a reasonable choice. To cross-check if this trend also holds for both other destinationaware protocols and a scenarios comprising devices i 2 Dt with temporary infrastructure access, we plot the delivery ratio of Prophet, MaxProp, and Spray&Focus as a function of a for single- and multicopy mode as well as P p ¼ 0:2; P t ¼ 0:4 and P n ¼ 0:4, respectively, in Fig. 4(c) and (f). Consistent with the findings described above, the impact of a is most clearly visible for Prophet, since routing in Prophet is based solely on destination-awareness, and suggests to choose a value of a ¼ 1. Recall that neither MaxProp nor Spray&Focus are pure destination-aware protocols. MaxProp uses multiple mechanisms to determine whether to forward a message, e.g., the number of hops the message has already traversed, and falls back to destination-awareness only to break ties. Therefore, the impact of a is less prominent, but again a value of a ¼ 1 seems to be a reasonable choice. In Spray&Focus the impact of a vanishes with a higher initial number of forwarding tokens L, as routing is destination-aware only in the focus stage employed once the number of forwarding tokens message reaches a replication count L ¼ 1. As we use a small ttl ¼ 1 h, the focus stage is reached less frequently for a higher initial replication count L, resulting in smaller influence of a. However, again a value of a ¼ 1 is a reasonable choice. We conclude from the figures shown in this section that it is generally beneficial to first route towards infrastructure for destination-aware protocols in hybrid DTNs, even with few infrastructure-capable devices. This implies that routing towards the infrastructure can be implemented independently of the destination, assuring scalability of this step.

5.3. Delivery performance Fig. 5(a) and (d) plot the delivery ratio as a function of the fraction of infrastructure-enabled devices x for P t ¼ 0 and ttl ¼ 1 h. All three protocols show a similar performance boost due to iHRS when increasing x. Recall that x ¼ 0 comprises the baseline of pure DTN routing with no devices being capable of accessing the infrastructure. As an example for the performance boost, Prophet in multi-copy mode with 30% devices with permanent infrastructure access boosts the delivery ratio from 0.24 to 0.65, i.e., by 169%, compared to pure DTN routing. This also reduces the delivery delay, i.e., the time elapsed between sending a message and reception of the first copy by the destination, shown in Fig. 5(b) and (e) as function of x, again for ttl ¼ 1 h. Here, e.g., that Spray&Focus in multicopy mode with 30% devices with infrastructure access reduces delay by 39%, from 19.5 min to 11.9 min. The almost linear increase of delivery ratio and the almost linear decrease of delivery delay shows that iHRS effectively utilizes the transmission opportunities provided by the infrastructure. We conclude from Fig. 5(a)–(e) that destination-aware protocols can heavily benefit from infrastructure in a hybrid DTN in a distributed setting. To close the loop to [11,13,12], we finally illustrate the impact of temporary infrastructure access provided by access points. Fig. 5(c) and (f) plot delivery ratio and delivery delay, respectively, as a function of the fraction of devices with temporary infrastruct ture y ¼ Pt PþP access for Prophet in multi-copy mode. To isolate the n effects, we set the number of devices with permanent infrastructure access to P p ¼ 0. Different geographic coverage of a single access point is considered, while access points are placed uniformly at random on the playground. Note that this may imply that areas covered by individual access points overlap. As a result, both the fraction of devices capable of infrastructure access and the coverage of the access points comprise degrees of freedom. Fig. 5(c) shows that increasing the fraction of devices capable of temporary

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

10

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx 35

0.6 0.4 Prophet MaxProp SprayFocus

0.2 0

20 15 10

0.2

0.4

0.6

0.8

1

0.2

0.4

0.6

0.8

1

0

Mean delivery delay [min]

Prophet MaxProp SprayFocus

0.2 0 0.2

0.4

0.6

0.8

1

Fraction of infrastructure-capable devices

0.4

0.6

0.8

1

30 Prophet MaxProp SprayFocus

30 25 20 15 10 5 0

0

0.2

Fraction of WiFi-capable devices

35

0.4

400 APs = 36% coverage 125 APs = 11% coverage 50 APs = 4% coverage

Fraction of infrastructure-capable devices

1

0.6

0.4

0 0

Fraction of infrastructure-capable devices

0.8

0.6

0.2

5 0

0

Delivery ratio

0.8

25

Mean Delivery delay [min]

Delivery ratio

0.8

1 Prophet MaxProp SprayFocus

30

Delivery ratio

Mean delivery delay [min]

1

25 20 15 10 400 APs = 36% coverage 125 APs = 11% coverage 50 APs = 4% coverage

5 0

0

0.2

0.4

0.6

0.8

1

Fraction of infrastructure-capable devices

0

0.2

0.4

0.6

0.8

1

Fraction of WiFi-capable devices

Fig. 5. Influence of fraction of infrastructure-capable devices on overall message delivery performance.

infrastructure access has higher impact on delivery performance that increasing the coverage. In the analyzed scenario, using a coverage of 4% and y ¼ 1 yields the same delivery ratio as 11% coverage with y ¼ 0:5 or 36% coverage with y ¼ 0:3. Fig. 5(f) shows that delivery delay also decreases with the number of infrastructurecapable devices and the number of access points. Again, increasing the number of access points reduces delay as much as enabling infrastructure access for the devices. e.g., increasing the coverage from 4% to 36% leads for y ¼ 0:2 to almost the same delay as increasing y from 0.2 to 0.8 with constant coverage. We conclude from Fig. 5(c) and (f) that increasing the number of infrastructurecapable devices is more efficient to boost performance of destination-aware protocols than increasing the coverage of the access points. 5.4. Storage and communication overhead Recall that DTN routing is always a trade-off between overhead and performance. To quantify overhead in terms of required storage, Fig. 6(a) and (d) plot the maximal message queue length of any device at any time during the simulation as function of x. The messages stored at proxy admins in case of multi-copy also contribute to the queue length. It can be observed that iHRS reduces storage requirements by delivering messages earlier if more devices are capable of accessing the infrastructure. However, messages stored at proxy admins cause an initial increase of storage overhead when increasing x. This is most obvious with Spray&Focus that uses heavy replication. Fig. 6(a) and (d) also illustrate the cost of routing towards the infrastructure first: without infrastructure-enabled devices (and, thus, no routing towards the infrastructure) the overhead is only little lower or in some case even higher than with few infrastructure-enabled devices, in particular in the multi-copy case. The fact that storage overhead for singlecopy is significantly lower than for multi-copy for all protocols and a low number of infrastructure-capable devices shows that running single-copy is a good choice with HRS. We conclude from

Fig. 6(a) and (d) that exploiting infrastructure access in hybrid DTNs can reduce storage requirements of destination-aware DTN protocols. Fig. 6(b) and (e) show the average number of ad hoc message communications per generated message as a function of x. For single-copy mode this number is decreasing up to the point where no DTN routing is performed at all and all messages are delivered across the infrastructure. Communication cost for multi-copy mode shows an initial increase due to stronger replication, before it decreases similar to single-copy mode. We conclude from Fig. 6(b) and (e) that exploiting infrastructure access in hybrid DTNs may increase the ad hoc communication cost slightly. However, we think that this cost is tolerable compared to the performance gains. Obviously, fewer ad hoc communication operations must result in increasing infrastructure communication. This is illustrated in Fig. 6(c) and (f), which plot the average number of messages exchanged across the infrastructure per generated message. For single-copy mode this measure increases strongly already with a small fraction of infrastructure-capable devices, resulting from routing preferably towards the infrastructure due to a ¼ 1. For multi-copy mode, multiple copies of a message may traverse the infrastructure, although a single copy may traverse it only once. This increases the load in the infrastructure, in particular for x 0:5 where many messages can be both replicated on the way towards the infrastructure and picked up by multiple infrastructure-capable devices. We conclude from Fig. 6(c) and (f) that heavy replication in multi-copy mode should be avoided for destinationaware protocols in hybrid DTNs. 6. Related work The basic motivation for out work is provided by Lindgren et al. [11], who empirically study the impact of communication infrastructure on DTNs. By analysis of real-world mobility traces they count device contacts when two devices are associated to the same WiFi or cellular network. They conclude that integration of

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

11

50 40 30 20 10 0 0

0.2

0.4

0.6

0.8

1

2.5 Prophet MaxProp SprayFocus

2 1.5 1 0.5 0 0

160 140 120 100 80 60 40

Prophet MaxProp SprayFocus

20 0 0

0.2

0.4

0.6

0.8

1

Fraction of infrastructure-capable devices

0.2

0.4

0.6

0.8

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

1

18 16 14 12 10 8 6 4

Prophet MaxProp SprayFocus

2 0 0

0.2

0.4

0.6

0.8

Fraction of infrastructure-capable devices

1

Prophet MaxProp SprayFocus 0

Fraction of infrastructure-capable devices

Ad-hoc comm. p. created message

Worst device queue [messages]

Fraction of infrastructure-capable devices

180

Infra. comm. p. created message

Prophet MaxProp SprayFocus

60

0.2

0.4

0.6

0.8

1

Fraction of infrastructure-capable devices

Infra. comm. p. created message

70

Ad-hoc comm. p. created message

Worst device queue [messages]

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

6 Prophet MaxProp SprayFocus

5 4 3 2 1 0 0

0.2

0.4

0.6

0.8

1

Fraction of infrastructure-capable devices

Fig. 6. Influence of fraction of infrastructure-capable devices on storage and communication overhead.

infrastructure significantly increases contact opportunities and decreases inter-contact times. By the same methodology, Hui et al. [12] find a phase transition: Adding more infrastructure after a certain point does not increase performance significantly any more. Banerjee et al. [13] analyzed the impact of different infrastructure components—WiFi hotspots, meshes, and relays—on DTN performance. They found that WiFi hotspots connected to an Internet infrastructure are costly, but allow for best performance enhancement due to shortcuts using a central server accessible to connected devices. KioskNet [15] allows Internet access for rural areas. Users are registered to kiosks that acts as gateway, with a truck transferring data from kiosks to Internet connection in larger cities. In our system the gateways are dynamic and represented by participating devices themselves. Therefore, our main challenge of finding appropriate gateways is solved in KioskNet through static registration of devices on kiosks—an approach not applicable in a dynamic distributed setting. For analyzing stability of such systems in terms of data being up/download, Galluccio et al. present in [29] an analytical framework. Castro et al. [30] evaluate DHT-based P2P protocols for use in opportunistic resource sharing with static infostations. While they focus on how to make the objects stored on mobile devices available through replication when those are disconnected, our goal is to provide communication between the mobile devices. Throwboxes introduced in [31] are static DTN devices with large capacity resources for storing messages when two devices miss each other when moving by the same site but at different times. The authors solve the placement of Throwboxes in DTNs through optimization in order to increase the bandwidth of the network. HRS solves the problem of missed contacts by storing messages in the overlay in a distributed manner. Inserting additional virtual nodes into an overlay similar to vHRS have been used in the context of DHT load balancing [32], however, without explicit placement of nodes. The indirection approach of iHRS is similar to i3 by Stoica et al. [25], making use of DHT-based registrations. Different to i3, iHRS stores messages in the overlay and uses further DTN-specific mechanisms.

7. Conclusion and outlook Integrating DTNs with opportunistic infrastructure access strongly boosts performance and allows for communication in heavily clustered scenarios. In this paper we presented a methodology for hybrid routing and our approach, called the Hybrid Routing System (HRS), that transparently integrates infrastructure encounters together with DTN routing into a decentralized routing system. For the class of destination-aware DTN protocols we developed two HRS schemes that are applicable for different time-scales of DTN contact stability. We integrated both schemes with three state-of-the-art DTN protocols and presented extensive evaluation results showing the performance of HRS. Our simulation study confirmed the observation that exploiting infrastructure can boost the performance of DTN routing protocols. In fact, we show that this is even true in a decentralized setting with many approximations that are necessary to guarantee scalability. To this end, the most efficient approximation is routing towards the infrastructure first instead of maintaining a proximity value for every device and the infrastructure. This can also be used to improve scalability in centralized settings. A further observation is that infrastructure reduces ad hoc forwarding in a DTN because it delivers messages quickly, in particular in single-copy cases. One may ask about the cost in terms of increased load for infrastructure capable devices. On the one hand, the load for message relaying may increase since these may be considered to be ‘‘better’’ forwarders—a fact that is also true for good forwarders in pure DTN routing protocols. On the other hand, the load may increase due to maintenance of the structured overlay. Although not explicitly considered here, we expect this load to be low, since only a few overlay connections must be maintained, which is compensated by the reduced ad hoc forwarding load in single-copy mode. In multi-copy mode, load for devices can be quite high for all devices independent of Internet connectivity, so the question is if such protocol should be employed on resource constrained devices at all. Future work includes integration of HRS into our ariba [33] overlay-library for developing networked application that currently requires continuous connectivity. We further explore

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018

12

C.P. Mayer, O.P. Waldhorst / Computer Communications xxx (2014) xxx–xxx

the benefits of communicating awareness information across the overlay similar to [34], and integrate new DTN protocols like RAPID [7]. Finally, acknowledgment schemes as analyzed by Thompson et al. [35] are interesting for HRS to employ in the overlay structure for storing IDs of delivered messages to reduce multi-copy message replication.

[18] [19]

[20]

References [1] M. Conti, M. Kumar, Opportunities in opportunistic computing, IEEE Comput. 43 (1) (2010) 42–50. [2] Z. Zhang, Routing in intermittently connected mobile ad hoc networks and delay tolerant networks: overview and challenges, IEEE Commun. Surv. Tutorials 8 (1) (2006) 24–37. [3] A. Chaintreau, P. Hui, C. Diot, R. Gass, J. Scott, J. Crowcroft, Impact of human mobility on opportunistic forwarding algorithms, IEEE Trans. Mobile Comput. 6 (6) (2007) 606–620. [4] A. Vahdat, D. Becker, Epidemic routing for partially-connected ad hoc networks, Technical Report CS-200006, Department of Computer Science, Duke University, Durham, NC, USA, Apr. 2000. [5] P. Hui, J. Crowcroft, E. Yoneki, BUBBLE rap: social-based forwarding in delaytolerant networks, IEEE Trans. Mobile Comput. 10 (11) (2011) 1576–1589. [6] A. Mtibaa, M. May, C. Diot, M. Ammar, PeopleRank: combining social and contact information for opportunistic forwarding, in: Proceedings of the IEEE International Conference on Computer Communications (INFOCOM) Mini Conference, San Diego, CA, USA, 2010, pp. 1–5. [7] A. Balasubramanian, B.N. Levine, A. Venkataramani, Replication routing in DTNs: a resource allocation approach, IEEE/ACM Trans. Netw. 18 (2) (2010) 596–609. [8] A.F. Lindgren, A. Doria, O. Schelén, Probabilistic routing in intermittently connected networks, ACM SIGMOBILE Mobile Comput. Commun. Rev. 7 (3) (2003) 19–20. [9] J. Burgess, B. Gallagher, D. Jensen, B.N. Levine, MaxProp: routing for vehiclebased disruption-tolerant networks, in: Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Barcelona, Spain, 2006, pp. 1–11. [10] T. Spyropoulos, K. Psounis, C. Raghavendra, Efficient routing in intermittently connected mobile networks: the multiple-copy case, IEEE/ACM Trans. Netw. 16 (1) (2008) 77–90. [11] A.F. Lindgren, C. Diot, J. Scott, Impact of communication infrastructure on forwarding in pocket switched networks, in: Proceedings of the ACM SIGCOMM Workshop on Challenged Networks (CHANTS), Pisa, Italy, 2006, pp. 261–268. [12] P. Hui, A.F. Lindgren, J. Crowcroft, Empirical evaluation of hybrid opportunistic networks, in: Proceedings of the International Conference on COMmunication Systems and NETworkS (COMSNETS), Bangalore, India, 2009, pp. 1–10. [13] N. Banerjee, M.D. Corner, D. Towsley, B.N. Levine, Relays, base stations, and meshes: enhancing mobile networks with infrastructure, in: Proceedings of the International Conference on Mobile Computing and Networking (MobiCom), San Francisco, CA, USA, 2008, pp. 81–91. [14] A.F. Lindgren, A. Doria, J. Lindblom, M. Ek, Networking in the land of Northern Lights – two years of experiences from DTN system deployments, in: Proceedings of the ACM Workshop on Wireless Networks and Systems for Developing Regions (WiNS-DR), San Francisco, CA, USA, 2008, pp. 1–8. [15] S. Guo, M. Derakhshani, H. Falaki, U. Ismail, R. Luk, E. Oliver, S.U. Rahman, A. Seth, M. Zaharia, S. Keshav, Design and implementation of the KioskNet system, Comput. Netw. 55 (1) (2011) 264–281. [16] C.P. Mayer, Hybrid routing in delay tolerant networks (Dissertation), Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany, 2011. . [17] T. Spyropoulos, K. Psounis, C.S. Raghavendra, Spray and wait: an efficient routing scheme for intermittently connected mobile networks, in: Proceedings

[21]

[22]

[23]

[24]

[25] [26]

[27] [28] [29]

[30]

[31]

[32]

[33]

[34]

[35]

of the ACM SIGCOMM Workshop on Delay Tolerant Networking (WDTN), Philadelphia, PA, USA, 2005, pp. 252–259. E. Daly, M. Haahr, Social network analysis for information flow in disconnected delay-tolerant MANETs, IEEE Trans. Mobile Comput. 8 (5) (2009) 606–621. S. Nelson, M. Bakht, R. Kravets, Encounter-based routing in DTNs, in: Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), Rio de Janeiro, Brazil, 2009, pp. 846–854. A. Mei, G. Morabito, P. Santi, J. Stefa, Social-aware stateless forwarding in pocket switched networks, in: Proceedings of the IEEE International Conference on Computer Communications (INFOCOM) Mini Conference, Shanghai, China, 2011, pp. 251–255. V. Erramilli, M. Crovella, A. Chaintreau, C. Diot, Delegation forwarding, in: Proceedings of the ACM Symposium on Mobile Ad hoc Networking and Computing (MobiHoc), Hong Kong, China, 2008, pp. 251–260. I. Stoica, R. Morris, D. Karger, M.F. Kaashoek, H. Balakrishnan, Chord: a scalable peer-to-peer lookup service for internet applications, in: Proceedings of the ACM Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM), San Diego, CA, USA, 2001, pp. 149–160. A. Rowstron, P. Druschel, Pastry: scalable, decentralized object location and routing for large-scale peer-to-peer systems, in: Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms (Middleware), Lecture Notes in Computer Science, Heidelberg, Germany, vol. 2218, 2001, pp. 329–350. F. Dabek, B. Zhao, P. Druschel, J. Kubiatowicz, I. Stoica, Towards a common API for structured peer-to-peer overlays, in: Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS), Berkeley, CA, USA, 2003, pp. 33–44. I. Stoica, D. Adkins, S. Zhuang, S. Shenker, S. Surana, Internet indirection infrastructure, IEEE/ACM Trans. Netw. 12 (2) (2004) 205–218. J. Karvo, J. Ott, Time scales and delay-tolerant routing protocols, in: Proceedings of the ACM Workshop on Challenged Networks (CHANTS), San Francisco, CA, USA, 2008, pp. 33–40. A. Keränen, T. Kärkkäinen, J. Ott, Simulating mobility and DTNs with the ONE, J. Commun. 5 (2) (2010) 92–105. F. Ekman, A. Keränen, J. Karvo, J. Ott, Working day movement model, in: Proceedings of the MobilityModels, Hong Kong, China, 2008, pp. 33–40. L. Galluccio, G. Morabito, A.L. Moustakas, S. Palazzo, Opportunistic communications in infostation systems: delay and stability analysis, in: Proceedings of the IEEE Global Telecommunications Conference (GLOBECOM), Miami, FL, USA, 2010, pp. 1–5. M.C. Castro, L. Galluccio, A. Kassler, C. Rametta, Opportunistic P2P communications in delay tolerant rural scenarios, EURASIP J. Wireless Commun. Netw. 2011 (2011) 9:1–9:14. W. Zhao, Y. Chen, M. Ammar, M. Corner, B. Levine, E. Zegura, Capacity enhancement using throwboxes in DTNs, in: Proceedings of the IEEE International Conference on Mobile Adhoc and Sensor Systems (MASS), Vancouver, Canada, 2006, pp. 31–40. A. Rao, K. Lakshminarayanan, S. Surana, R. Karp, I. Stoica, Load balancing in structured P2P systems, in: Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS), Lecture Notes in Computer Science, Berkeley, CA, USA, 2003, pp. 68–79. C. Hübsch, C.P. Mayer, S. Mies, R. Bless, O.P. Waldhorst, M. Zitterbart, Reconnecting the Internet with ariba: self-organizing provisioning of end-toend connectivity in heterogeneous networks, ACM SIGCOMM Comput. Commun. Rev. 40 (1) (2010) 131–132 (an earlier version appeared in Proceedings of the ACM SIGCOMM 2009, Barcelona, Spain). M.P. Wittie, K.A. Harras, K.C. Almeroth, E.M. Belding, On the implications of routing metric staleness in delay tolerant networks, Comput. Commun. 32 (16) (2009) 1699–1709. N. Thompson, S.C. Nelson, M. Bakht, T. Abdelzaher, R. Kravets, Retiring Replicants: congestion control for intermittently-connected networks, in: Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), San Diego, CA, USA, 2010, pp. 1–9.

Please cite this article in press as: C.P. Mayer, O.P. Waldhorst, Routing in hybrid Delay Tolerant Networks, Comput. Commun. (2014), http://dx.doi.org/ 10.1016/j.comcom.2014.03.018