MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios

Ad Hoc Networks xxx (2012) xxx–xxx

Contents lists available at SciVerse ScienceDirect

Ad Hoc Networks journal homepage: www.elsevier.com/locate/adhoc

MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios Donato Macone a, Guido Oddi b,⇑, Antonio Pietrabissa b a b

University of Rome Sapienza, Department of Information, Electrical and Telecommunication Engineering (DIET), via Eudossiana 18, 00184 Rome, Italy University of Rome Sapienza, Department of Computer, Automatic and Management Engineering ‘‘Antonio Ruberti’’, via Ariosto 25, 00185 Rome, Italy

a r t i c l e

i n f o

Article history: Received 25 November 2011 Received in revised form 6 September 2012 Accepted 27 September 2012 Available online xxxx Keywords: Mobile-Ad-Hoc-Networks Markov Decision Process Reinforcement Learning Routing Energy Mobility Link availability

a b s t r a c t Mobile-Ad-Hoc-Networks (MANETs) are self-configuring networks of mobile nodes, which communicate through wireless links. The main issues in MANETs include the mobility of the network nodes, the scarcity of computational, bandwidth and energy resources. Thus, MANET routing protocols should explicitly consider network changes and node changes into the algorithm design. MANETs are particularly suited to guarantee connectivity in disaster relief scenarios, which are often impaired by the absence of network infrastructures. Moreover, such scenarios entail strict requirements on the lifetime of the device batteries and on the reactivity to possibly frequent link failures. This work proposes a proactive routing protocol, named MQ-Routing, aimed at maximizing the minimum node lifetime and at rapidly adapting to network topology changes. The proposed protocol modifies the Q-Routing algorithm, developed via Reinforcement Learning (RL) techniques, by introducing: (i) new metrics, which account for the paths availability and the energy in the path nodes, and which are dynamically combined and adapted to the changing network topologies and resources; (ii) a fully proactive approach to assure the protocol usage and reactivity in mobile scenarios. Extensive simulations validate the effectiveness of the proposed protocol, through comparisons with both the standard Q-Routing and the Optimized Link State Routing (OLSR) protocols.  2012 Published by Elsevier B.V.

1. Introduction Mobile-Ad-Hoc-Networks (MANETs) are self-configuring networks of mobile nodes which communicate by wireless links. Since the network topology continuously varies due to the node mobility, the main challenge in MANET management is to allow each node to correctly route the packets to the other nodes. Beside the mobility of nodes, the routing algorithms must face other specific and demanding challenges, such as the scarcity of computational and bandwidth resources and the energy consumption limitations given by the battery of the connected devices. Therefore, it is crucial that MANET routing ⇑ Corresponding author. E-mail addresses: [email protected] (D. Macone), [email protected] (G. Oddi), [email protected] (A. Pietrabissa).

protocols consider network changes and node changes into the algorithm design itself, by explicitly taking into account information on mobility, available bandwidth and residual energy. MANETs are particularly suited in disaster scenarios, since they do not rely on fixed infrastructures. In this respect, the work presented in this paper was carried out in the EU funded project MONET [1], where a hybrid MANET-satellite network is being developed to disaster relief purposes. The rescue scenarios, as the ones encompassed in MONET, entail more severe requirements with respect to other less critical scenarios. In fact, in rescue scenarios, (i) all the nodes should have long and homogeneous lifetimes, i.e., no nodes should have discharged batteries during the rescue mission, leading to the isolation of a member of the rescue team and to the impossibility of exchanging vital information, (ii) the communications

1570-8705/$ - see front matter  2012 Published by Elsevier B.V. http://dx.doi.org/10.1016/j.adhoc.2012.09.008

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

2

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

should be resilient to link failures due to the mobility of the nodes. In the development of the routing algorithm, GPS and energy level information are assumed to be exchanged among MANET devices, in compliance with the specifications of the mobile nodes developed within MONET project. Therefore, the main concept of the proposed algorithm is that the routing strategy consists of choosing the next-hop node in order to use the most energy-rich and stable path: based on the residual energy of the network devices, the algorithm aims at increasing the minimum lifetime of the nodes by balancing the battery usage among all the network nodes; moreover, the algorithm takes into consideration the mobility of each node in choosing the route towards a destination, in order to rapidly adapt to networks changes. The proposed routing algorithm is based on the Q-Routing algorithm [2], developed for fixed networks, and extends it to cope with critical mobile scenarios; therefore, the algorithm is named Mobile Q-Routing (MQ-Routing). In particular, MQ-Routing is based on Reinforcement Learning (RL) methodologies: the network routing problem is modelled as a Markov Decision Process (MDP), which is then solved with a Q-learning algorithm [3] – a wellknown RL technique. 1.1. State of the art and proposed innovations Routing in MANETs is, along with other challenging issues in the management of ad hoc networks (e.g. distributed congestion control [4], load balancing [5] and admission control [6]), a widely studied research topic, and the problem has been dealt with by a plethora of approaches (see for instance [7,8] and the references therein). Three main classes of routing algorithms are studied: proactive, reactive and hybrid algorithms. Proactive algorithms aim at setting up in advance all source–destination paths within the network, regardless the necessity of forwarding a data traffic flow to a specific destination node. In reactive algorithms, source–destination paths are calculated on-demand, i.e., as soon as a data traffic flow needs to be transmitted. Hybrid algorithms jointly use both the approaches. Each class of routing algorithms provides particular advantages and drawbacks. Proactive algorithms are characterized by a low setup path delay, since the routing information are already available as soon as a new flow arises, and are more suitable for Quality of Service (QoS) extensions (see for instance [9]). The main drawback is the increase of the control overhead necessary to maintain an updated vision of the whole network. On the counterpart, reactive algorithms experience a higher path setup time but reduce the control packets exchange thanks to the on-demand approach, especially at low speeds. Several studies have been carried out, in literature, in the field of proactive and reactive routing protocols in MANET. In particular, there are two protocols that are widely used on the field and are usually considered as benchmarks for subsequent algorithms: the Optimized Link-State Routing Protocol (OLSR) [10] and the Ad hoc On-Demand Distance Vector Protocol (AODV) [11]. The OLSR protocol is a proactive protocol based on the link state approach [7]. Each node internally generates a

network connectivity graph and associates a specific weight to each link of the graph representing its current state. Each node independently calculates the best route towards each feasible destination. OLSR uses flooding to disseminate control and topology information to all network nodes (Hello and Topology Control messages). In order to minimize the amount of control overhead flooded throughout the network, OLSR uses the concept of Multipoint Relays (MPRs) nodes. MPRs are selected in such a way that each node is able to reach all its two-hop neighbours by means of a MPR. Hello messages are specific messages that each node periodically broadcasts to its neighbours, containing information about the set of neighbours the node is able to reach. Through these messages, each node is able to generate a vision of its two-hop neighbours in order to properly select the MPRs. The MPR nodes periodically broadcast Topology Control messages to the other MPR nodes, providing the network nodes with updated link-state information in order to allow internal route calculation. Topology Control messages are forwarded only within the set of MPRs, reducing the total control overhead of the network. AODV is a reactive routing protocol based on the distance vector approach [8]. In such an approach each node is not aware of the whole network topology, but it associates to each destination one value representing the distance to reach the destination, expressed by a proper metric (e.g., number of hops or total experienced delay), and the indication of the related next-hop. AODV avoids the loop formation, which occurs in MANETs due to variability of the network graph (see [12,7]). Signalling includes the broadcasting of specific Route Request (RREQ) and Route Reply (RREP) messages, which generate the requested source–destination path, and Route Error (RERR) messages for link failure management purposes. Similarly to the OLSR, the proposed algorithm follows a proactive approach, steadily updating network control information in a distributed fashion. However, the proposed algorithm also presents several similarities with the AODV protocol, since the approach is similar to distance vector and Path Error Packets (see Section 3.4) are used to fast disseminate path changes, frequent in MANET scenarios. The proposed algorithm, as widely exposed in the following sections, aims at increasing network nodes’ lifetime by proper energy-efficient policies and achieving fault-tolerance, through a rapid path-handover and mobility prediction (along with the improvement of the application throughput). The energy efficiency is an issue caused by the constraints imposed by the battery capacity. In a MANET, there are different ways to increase energy efficiency. One of them is the possibility of dynamically offloading the computation from the local terminal to remote, energy rich nodes (e.g., fixed servers [13]). This approach is not suited in the critical scenarios considered in the MONET project, in which no centralized nodes are available and the computation effort is low compared to the data transmission task. Another choice is to design a strategy to switch on/ off radio transmitters in order to conserve energy. In [14], for instance, the radio is turned off on the basis of

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

application level information. Xu et al. [15] saves energy by identifying the nodes that are equivalent from a routing perspective and then turning off the unnecessary ones. In the scenarios as the ones considered in the paper, in which vital information must be exchanged between rescuers at any time, it is unacceptable to turn off the transmitters for a potentially high amount of time. A third approach is based on proper topology control techniques, able to adjust the transmission range of transmitters to create a desired topology. Xiang et al. [16] proposes a routing protocol in which the network topology is dynamically optimized in reference to the real-time node energy consumption, expressed as a specific cost function along the transmission link. Konstantinidis et al. [17] uses a combination of problem-specific light-weighted local search and genetic algorithm, in order to solve the minimum energy network connectivity (NP-HARD problem) in an approximated fashion. Bahramgiri et al. [18] controls the topology of a multihop wireless network by varying the transmission power at each node, to preserve the connectivity of the network upon failing of a constant sub-set of nodes and to simultaneously minimize the transmission power at each node. The topology control approaches could lead to a consumption of considerable energy to maintain the topology due to node mobility: therefore, it is not suitable for the scenarios considered in this paper, where the nodes are characterized by a medium node mobility (up to 15 m/s) (see for instance [19]). Another possibility is to modify or enrich existing network protocols (e.g., MAC, routing and transport protocols), to include the energy consumption problem in the protocol design. In this respect, a plethora of approaches are available in literature. Benslimane et al. [20] reports an extension to OLSR protocol with the aim of selecting an appropriate set of MPRs, taking into account the link transmission power. This approach is different from MQ-Routing, because the authors of [20] do not take into account node residual energy but only the transmission power. Jing and Lee [21] proposes similar metrics based on residual energy and transmission power, but it applies these metrics to an extension to AODV which is a reactive routing protocol and is not suitable for critical scenarios (see Section 3.2 for further details). Regarding the fault tolerance property of routing algorithms, several papers have been published in literature. One approach to obtain fault-tolerance is by redundancy: the same application flow is replicated onto multiple paths (see [22] for instance). This approach is naturally not recommended in energy-constrained network scenarios, in which a raw replication can rapidly lead to the discharge of nodes battery. A second approach is to maintain a backup path (directed to every destination), to be rapidly utilized by the routing protocol in case of failure of the main path [23,22]. The main disadvantage is the difficulty to maintain a backup path in high mobility scenarios, which should also be disjoint from the main one, to increase fault tolerance (trying to avoid that a single link failure generates the fault of both the path). Our approach tries to maintain more than one backup path in memory (by using the Q-values related to each neighbour – see the following sections for details) and therefore to increase the fault tolerance (even if some of those paths are over-

3

lapped). A third approach is to design specific topologies to guarantee a high level of fault tolerance. Cardei et al. [24], for instance, uses topology control in order to support sensor data reporting even in the event of failures of a subset of nodes. This approach is more suitable for fixed networks, in which topology maintenance is easier. None of the cited papers provides a holistic approach, valuable for energy-constrained and mobile ad hoc networks, which is the aim of this work. Regarding the application of Reinforcement Learning (RL), several studies have been conducted. The Q-Routing protocol [2,25] applies the Q-learning algorithm [3] to the shortest-path routing problem in fixed-topology networks. Several enhancements to the basic Q-Routing algorithms have been proposed. In [26], a Q-Routing approach aimed at minimizing the packet delivery time is proposed; also this approach is however suitable for fixed networks. In [27], the convergence of the Q-Routing algorithm is speeded up by means of a Least Squares Policy Iteration (LSPI); however, it takes into account fixed networks in which only the link availability changes in time, and does not study in detail the loop formation problem. Marbach et al. [28] uses Q-Learning to design an algorithm performing both call admission control and routing; however, this protocol does not take into account minimum node lifetime due to battery constraints. In [29], the stability route problem is considered in terms of route lifetime using a proactive approach, but it does not consider energy consumption of the nodes and do not take into account the loop formation problem. The proposed algorithm is based on the RL approach of the Q-Routing [2], detailed in Section 2, and reports/introduces some modifications, described in Section 3, to cope with critical MANET scenarios, taking into account simultaneously mobility, node energy consumption and link stability. Section 4 presents the simulation results, and in Section 5 the conclusions are drawn.

2. Background 2.1. Reinforcement Learning (RL) Under the Markovian1 and stationarity assumptions, a discrete-time Markov Decision Process (MDP) is defined by: 1. a finite state space S = {si}i=1,. . .,n; 2. a finite and non-empty set of available control actions Aðsi Þ ¼ fak gk¼1;:::;jAðsi Þj associated to each state si e S; 3. a real-valued one-step reward function r: S  A ? R, where r = r(si,ak) is the reward incurred by the system as it is in state si e S and action ak e A(si) is chosen; 4. the probability p(si,sj,ak) that, in the next time step, the system will be in state sj e S when action ak e A(si) in state si e S is chosen; the set of these transition probabilities constitute the transition matrix T.

1 A stochastic process has the Markov property if the conditional probability distribution of the next state of the process depends only upon the present state and is conditionally independent of past states.

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

4

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

In the following, we will denote with s(t), a(t), and r(t) the state, the action and the reward of the system at time t, respectively. A stationary policy is a function p: S ? A, which maps every state si e S to a unique control action ak e A(si). When the system operates under policy p, the MDP reduces to a discrete-time Markov chain, and the following expected discounted reward is earned by the system:

( ) t X 1 0 Rp ¼ lim sup Ep ct 1 rðt0 Þ ; t n!1 t 0 ¼1

ð2:1Þ

where 0 6 c 6 1 is the discount rate and the subscript p specifies that the controller operates under policy p. The MDP problem is the determination of the optimal policy p minimizing cost (2.1). Beside standard Dynamic and Linear Programming approaches [30], MDPs can be solved also via RL algorithms [3]: even if RL approaches achieve the optimal solution only under given conditions, such as infinite number of visits of each state, in practice they achieve approximate solutions to problems which are intractable for other methods. In our scenario, the RL approach was chosen since the Dynamic and Linear Programming approaches are not suited, mainly because (i) they need complete knowledge of the system statistical characteristics (i.e., the elements of transition matrix T); (ii) they cannot cope with non-stationary scenarios. RL algorithms are based on the observations of transition occurrences and gained rewards: in a RL task, in which the agent and the environment interact at discrete time steps, at time t, the agent observes state s(t) and produces an action a(t); then, it receives the reward r(t + 1) and observes the environment to infer the new state s(t + 1). The agent’s objective is to learn an optimal policy p, such that the discounted return (2.1) is maximized. Fig. 1 (see [3]) shows the described concept. The Q-learning algorithm, a simple and effective RL algorithm, is now briefly described. Under a given policy p, each pair (si, ak), si e S, ak e A(si), has an associated action-value function Qp(si, ak), which represents the reward to execute action ak in state si, summed to the expected reward achieved by following policy p, starting from the next state. The Q-learning learns the action-value functions online, with the following update rule:

Q ðsðtÞ; aðtÞÞ

QðsðtÞ; aðtÞÞ þ a½rðt þ 1Þ þ c max

ak 2Aðsðtþ1ÞÞ

Q ðsðt þ 1Þ; ak Þ

 QðsðtÞ; aðtÞÞ;

ð2:2Þ

where a is the learning rate of the agent, which has to be tuned based on the specific application scenario and allows State s(t) Reward r(t)

Agent

Action a(t)

to account for non-stationary scenarios. The convergence to the optimal policy is guaranteed if 0 6 c < 1 and all actions and state are ‘‘sufficiently’’ visited [3]. Given the estimate (2.2) of the action-value function Q(si, ak), an associated policy can be computed in different ways. The ‘greedy’ policy is determined by selecting in each state si e S the best (greedy) action, i.e. the action ak associated with the larger value according to the actionvalue function. However, to favour the learning process of RL, sometimes different actions should be selected in order to explore the state space; for this purpose, some algorithms use the so-called e-greedy policy, determined by selecting in each state si e S the greedy action with probability 1  e, and a random action with probability e, with 0 < e 6 1:

aegreedy ðsi Þ 8 < arg max Qðsi ;ak Þ;with probability 1  e ak 2Aðsi Þ ¼ ; 8si 2 S: ð2:3Þ : random action in Aðsi Þ;with probability e The tuning of the parameter e is needed to balance exploration (i.e., random action selection) and exploitation (i.e., greedy action selection based on the current best known policy). The Q-learning approach consists in learning the actionvalue function Q, and therefore the optimal policy, by experience, following proper update rules, starting from a given initial Q function (e.g., with all values set to 0). 2.2. Q-Routing As already mentioned, the proposed routing technique is an extension of Q-Routing [2,25]. In the Q-Routing algorithm, the network is modelled as a graph G = (V, E), where V is the set of nodes with cardinality |V| = n and E is the set of edges. Each node i e V is connected to a set of neighbour nodes Ni. The network routing problem is formulated as a MDP, considering a single packet coming from a source s and directed to a destination d: 1. S = {si = di | i e V}, where di is a vector of n elements equal to 0 but the ith element equal to 1, is the finite state set; each state indicates the (unique) current position of the packet; S 2. A = i=1,...,NA(si) is the action set, where A(si) = {aij|j e Ni)} is the action set of state si, and aij represents the decision of the agent to move from state si to state sj (i.e., node i sends the packet to neighbour j); 3. the one-step reward obtained performing action a(t) = aij in state s(t) = si to reach state s(t + 1) = sj is 1 only if at time t + 1 the packet is in the destination node d, it is 0 otherwise; the reward is then function of the reached state sj only:

rj ¼ s(t+1) r(t+1)

Environment

Fig. 1. Reinforcement Learning task.



1 j¼d 0 otherwise

:

ð2:4Þ

The transition matrix is not defined, since RL algorithms implicitly estimate the transition probabilities within the action-value function estimation (2.2).

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

5

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

The routing algorithm runs as follows. Each node i e V keeps in memory a matrix of elements S Q i ðd; jÞ 2 ½0; 1njAðsi Þj . Q = ieNQi is the action-value function: in node i, the routing decision (i.e., the next-hop j e Ni to reach destination d e V) is taken following an e-greedy policy, i.e., by choosing the neighbour such that j ¼ arg max Q i ðd; lÞ with probability (1  e) (exploitation), l2NðiÞ

or randomly with probability e (exploration); then, the system moves to state sk. The update procedure (2.2) occurs every time a packet is delivered to a neighbour node, and requires the neighbour which receives the packet from node i to send back an acknowledge packet (ACK). Note that this definition of action-value function implicitly allows a distributed implementation of the algorithm, since (i) each node i is responsible of updating its own part Qi of the value function, and (ii) the update rule (2.2) requires data from the neighbour nodes Ni only. If the network topology is stationary, as the update (2.2) of the Q function keeps going on, an estimation of the real state of the network is achieved: thanks to definition (2.4), each element Qi(d, l) represents the discounted reward incurred by the packet to reach node d e V when it is in node i following a path through neighbour j e Ni. The Q-Routing algorithm has the desired property of converging to the shortest path based on the hop-count metric: due to the fact that c < 1, the longest path has the smallest action-value; furthermore, the random exploration entailed by the e-greedy action selection allows to effectively update the action-values. 3. MQ-Routing algorithm By analyzing the Q-Routing behaviour in mobile applications and, in particular, in critical scenarios as the ones considered in the MONET project, the following main drawbacks were identified: 1. no resource usage is taken into account, such as node residual energy or bandwidth; 2. no adaptability to variable scenarios and slow reactions to path errors due to link failures: the e-greedy random exploration is not sufficient to follow rapid network changes due to node mobility and changes of node/network resources; 3. ACK messages generates an unacceptable amount of overhead; 4. loop management (i.e., the detection and resolution of loops in paths followed by packets), crucial in mobile networks, is not considered. The modifications of the Q-Routing algorithm aim at solving the listed drawbacks by reporting and introducing a set of appropriate extensions. In particular, Section 3.1 addresses the first point by defining a variable discount factor, Section 3.2 addresses the second and third points by relying on a proactive approach, and Section 3.3 the fourth point by introducing a detection-based loop management. Section 3.4 summarizes the MQ-Routing algorithm, whereas, in Section 3.5, the algorithm properties are described.

3.1. Definition and implementation of new proposed metrics To take into account the resource usage and the mobility during the routing process, proper metrics are reported and proposed in this Section. The idea follows the one introduced in Wu et al. [25], which defines a time-varying discount factor c = c(t). In MQ-Routing, to account for nonhomogeneous characteristics of the network links and nodes, different time-varying discount factors are associated to different nodes. The update rule (2.2) is modified accordingly:

Q i ðd; jÞ

Q i ðd; jÞ þ a½r j þ cij ðtÞmax Q j ðd;lÞ  Q i ðd; jÞ; l2NðjÞ

ð3:1Þ

where a (defined in Section 2.1 as the ‘‘learning rate’’) indicates, roughly speaking, the weight of the recent information with respect to older one. Such parameter a assumes values in the interval [0, 1]: the smaller the a value is, the greater importance is given to the older information; on the counterpart, the bigger the a value is, the greater importance is given to the upcoming information. In Eq. (3.1), cij(t) is the value of the discount factor at time t, when the system is in state s(t) = si and moves to state s(t + 1) = sj under action a(t) = aij. The initial values of the state-value function are assumed to be equal to 0. Three different metrics are reported in this Section, taking into account the link availability prediction, the residual energy of the nodes and the node mobility; the metrics are then combined together to yield the time-varying discount factor. A link availability metric is proposed and described in Section 3.1.1. It is computed by assuming that the nodes are GPS-enabled, and is used to compute the link availability factor cGPS ij ðtÞ. The mobility and residual energy metrics, detailed in Sections 3.1.1 and 3.1.2, lead to the computation of the mobility factor cMOB ðtÞ (which follows the idea proj posed in [25]) and of the energy factor cEN j ðtÞ, respectively. Note that (i) the energy and mobility factors are per-node factors, whereas the link availability factor is a per-link factor (involving the couple of nodes originating the link), and that (ii) together, the link availability and the mobility metrics provide information on the link stability (i.e., the stability of the communication between two nodes), based on their relative position and speed (link availability metric) and on node j behaviour (mobility metric). The overall discount factor from state si and state sj is computed as: MOB cij ðtÞ ¼ cMAX  cGPS ðtÞ  cEN ij ðtÞ  cj j ðtÞ;

ð3:2Þ

EN MOB By normalizing all the factors cGPS ðtÞ beij ðtÞ; cj ðtÞ and cj

tween 0 and 1 and by setting cMAX e [0, 1) as the maximum discount factor (reached when all the factors are equal to 1, i.e., with full energy and fixed nodes), the algorithm is guaranteed to tend towards convergence, as mentioned in Section 2.1; clearly, since the factor (3.2) is variable, the convergence paths change in time. The whole discount factor cij(t) defined in (3.2) is used in Eq. (3.1) to update Qi(d, j), i.e., the part of the action-value function which is handled by node i. According to Eq. (3.2), each single

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

6

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

metric gives a contribution to the total metric in relation to its current importance. For instance, in a relatively high mobile scenario, initially the values which are likely to be not close to 1 are the mobility and link availability factors; later, with the decrease of the residual energy of the nodes, also the energy factor value becomes substantial and the energy metric starts to gain importance in the routing procedure. Since MQ-Routing is a distributed algorithm (property inherited from the Q-Routing algorithm), each neighbour node j e Ni must communicate some information to node i to let it properly compute the variable discount factor, as described in the following two sections. 3.1.1. Link stability metrics As already mentioned, to take into consideration the stability of a link associated to a node couple, two distinct metrics are considered: the GPS-based link availability metric, proposed in this paper, and the mobility metric. The link availability metric uses the concept of ‘neighbour reachability time’, as defined in [31]. This time is calculated as follows. Let [xi(t), yi(t)] and [xj(t), yj(t)] be the positions at time t of nodes i and j, respectively, on a previously defined Cartesian coordinate system. Let also vi(t) and vj(t) be the velocities of the nodes at time t, hi(t) and hj(t) be their moving directions and TX be their transmission range (i.e., the maximum communication distance between two nodes). Then, the amount of time Tij(t) the mobile nodes i and j will stay connected, given the GPS information received at instant t, is given by:

T ij ðtÞ ¼

ðab þ cdÞ þ

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ða2 þ c2 ÞTX 2  ðad  cbÞ2 ða2 þ c2 Þ

;

ð3:3Þ

where a = vi(t)cos[hi(t)]  vj(t)cos[hj(t)], b = xi(t)  xj(t), c = vi(t)sin[hi(t)]  vj(t)sin[hj(t)] and d = yi(t)  yj(t). Once such a neighbour-reachability time is calculated, the time-varying discount factor is generated in the following way:

c

GPS ij ðtÞ

¼

8 qffiffiffiffiffiffiffiffiffiffi < T ij ðtÞ

T ij ðtÞ < T HELLO

:1

T ij ðtÞ P T HELLO

T HELLO

:

ð3:4Þ

The square root in Eq. (3.4) (and also in the following defMOB initions of cEN ðtÞ) is introduced since, if the j ðtÞ and cj plain ratio between Tij(t) and THELLO was considered, low values of cGPS ij ðtÞ would excessively lower the whole discount factor (3.2). In practice, Eq. (3.4) is representative of the probability that the link between nodes i and j is available THELLO seconds after the reception of a Hello Packet (see Section 3.2 for further details about the proactive approach). In such a way, links which are supposed to be stable in the next period will be chosen with higher probabilities. To let node i compute the link availability factor (3.4), each neighbour node j e Ni must communicate its position, speed and direction to node i. The main goal of metric (3.4) is to avoid the loss of a consistent portion of traffic in the time necessary to manage a link failure due to node mobility (which is handled by means of Path Error and Hello messages as defined in the

following sections) by changing the path before the link failure occurrence. The assumption under Eq. (3.3) is that nodes have simple mobility patterns (for example, quite-constant speed and no sudden changes of direction with brusque accelerations). In the considered scenario, the movement of a team of rescuers is the most of time in groups towards sensible objectives at quite constant speed with small accelerations, so that the model can be applied with sufficient accuracy. However, to enhance the link stability estimation, and to broaden the addressed situations and scenarios, another metric is considered, which accounts for peculiar node mobility patterns. The mobility metric was proposed in [25] and takes into consideration the mobility of nodes in terms of neighbour sets. Let ADB denote the symmetric difference between two sets A and B, and A [ B denote the union of these sets. The mobility factor is then calculated as the percentage of neighbours which remains the same between the sending of two consecutive Hello Packets: MOB ðtÞ j

c

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi jNj ðtÞDNj ðt  T HELLO Þj ; ¼ 1 jNj ðtÞ [ N j ðt  T HELLO Þj

ð3:5Þ

where with little abuse of notation, Nj(t) is the set of neighbours of node j at time t and THELLO is the hello interval (described in Section 3.2). The mobility factor (3.5) is computed by each node and sent to its neighbours; node i is then capable of computing the discount factor (3.2) since it receives the value of the mobility factors of each candidate next-hop nodes j e Ni. The main importance of the mobility metric is to let the routing algorithm prefer more stable nodes, with a high value of mobility factor, which are not likely to rapidly change their neighbour sets. 3.1.2. Energy metric The proposed MQ-Routing algorithm considers also the node residual energy as a routing metric:

cEN j ðtÞ ¼

qffiffiffiffiffiffiffiffiffiffi ej ðtÞ;

ð3:6Þ

where ej(t) is the available energy in node j at time t. The value of the factor (3.6) is communicated by each node to its neighbours, so that each node is able to compute the discount factor (3.3) for all its candidate next-hop nodes. The main objective of the metric is to balance the traffic through the network nodes in order to increase the minimum lifetime of the nodes. As above-mentioned, in rescue scenarios, all nodes should have high and homogeneous lifetimes: no nodes should have discharged batteries, leading to the impossibility of communicating vital information. By using the proposed metric, the chosen next-hop j is aimed at using the most ‘‘energy-rich’’ path. 3.2. Proactive approach To adapt to network changes regardless of the application flows currently active in the network and to decrease the overhead, the update rule (2.2) and the exploration are

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

performed in a proactive fashion (e.g. similar to OLSR), upon the receipt of a Hello Packet (see Section 3.4 for details) so that ACKs are no longer needed. Hello Packets are exchanged among neighbours in a 1-hop fashion (other approaches could be 2-hop [32] and, more generally, multi-hop [33] information exchange). The interval THELLO between two hello messages is the time interval between two consecutive transmissions of Hello Packets and addresses the trade-off between overhead and reactivity. At each time step, each node broadcasts to its neighbours a Hello Packet containing (i) its maximum estimates of the action-values towards each destination node, and (ii) all the information needed to calculate the variable discount factor, as detailed in Section 3.1. This periodic message exchange gives the algorithm the following advantages with respect to the Q-Routing: 1. reduction of the number of protocol control messages, since no per-packet ACK messages are needed and no random exploration is foreseen (as in e-greedy policy instead); 2. adaptation to variable scenarios, since the proactive approach maintains the action-values as much up to date as possible by periodic exploration, whereas the e-greedy policy relies on a random exploration, which is unable to rapidly follow network changes without causing an unacceptable throughput decrease. There are two cases in which a Hello Packet could not be able to reach a neighbour: (i) the packet is lost over the network or (ii) the neighbour is no longer reachable. In a network scenario with a high probability of packet losses, it is possible that a Hello Packet is lost before reaching a neighbour (as stated in case (i)). In this case, the affected neighbour continues to use old information about the status of the source of the Hello Packet during the following THELLO time period. After the expiration of the following THELLO, a new Hello Packet is forwarded, and the neighbour is able to update the routing information accordingly. Nevertheless, if consecutive Hello Packets from the same node get lost during an entire Neighbour hold time interval (see Table 2), the neighbour is considered as no more reachable and, consequently, discarded from the neighbour set Ni even if it is still reachable: this result is actually correct because in this case the network is not able of guaranteeing a reliable communication between the two nodes. The second case in which a Hello Packet is not able to reach the neighbour is when the neighbour is no more reachable from the source node of the Hello Packet. In this case, after the expiration of the Neighbour hold time interval, the source node of the Hello Packet will be discarded from the neighbour set Ni of its old neighbours. In both cases no instability issues arise from the unexpected loss of Hello Packets, even if an excessive loss of Hello Packets could lead to an increase of the time needed for the algorithm to converge (just as it happens in case of large ACK packets loss). A proactive approach was also chosen to decrease the path setup delay of each flow. In fact, especially in rescue scenarios, critical communications among users must be setup immediately at the request. These communications have the following characteristics: they are typically short,

7

provide vital information to other rescue groups, and consist of both voice and video/data communications. A reactive approach would lead to unacceptable delays. To speed up the dissemination of path changes, an approach similar to AODV [11] and DSDV [12] is considered, which uses Path Error messages: as soon as a link failure happens (either due to node mobility or due to neighbour node energy discharge) and the previously connected nodes realize that a known destination is not reachable anymore, the nodes flood a Path Error message to alert network nodes about the link failure. In such a way, a faster update of the network state is performed and, thus, faster reactions to network changes are possible. 3.3. Loop management In common distance vector protocols for MANETs, such as AODV [11] and DSDV [12], the path towards a destination is unique and univocal and a sequence number, representing the obsolescence of a path, can be directly associated to it. In MQ-Routing, in contrast, several overlapping paths to a destination can coexist, since the best paths vary over time (according to the node mobility, to the node discharging occurrences and to the time-varying discount factors), leading to the difficulty to manage loops with mono-dimensional sequence numbers. Thus, in MQRouting, loops are managed in a detection-based way: as a loop is generated in a node n towards a destination d because of a link failure or of a link status modification (e.g., because the residual energy of the peer node changes), the same node executes the following steps to manage and solve the loop (see Section 3.4 for packets details): 1. as soon as a loop is detected in node n, through the inspection of the sequence number included in the Data Packet, it sends a Probe Packet towards destination d, following the path determined by the current routing tables; 2. if the Probe Packet reaches again node n, then a Path Reset message is forwarded following the reverse loop path determined by the Probe Packet journey (this reverse path represents a loop); 3. when a loop node receives the Path Reset message, it resets to zero the associated action-values and broadcasts new next-hop n0 , if available, and the associated Qs(d,n0 ), to speed up the update of routing tables. 4. When the path reset message reaches the node generating the loop, or the Loop Recovery timer expires, the loop management procedure is terminated. The loop recovery timeout (defined by the Loop Recovery timer) determines the time of the whole loop recovery procedure, i.e., the time from the transmission of the first Probe Packet to the actual reset of the routing table information (inside the loop nodes) which caused the loop. If this loop recovery timeout expires there are two possibilities: (i) the loop was solved, e.g., through the link re-connection, or (ii) one or more Probe Packets were lost during the loop procedure. The second case is a case of interest, especially in mobile wireless network: under the

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

8

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

condition that Probe Packets will eventually reach all the nodes involved in the loop, the loop will be solved (since this procedure is triggered at every loop detection instance). The loop recovery timeout should be tuned considering two main aspects: (i) it must not be too short, since it could lead to the impossibility to solve loops; (ii) it must not be too long, since it could lead a node to indefinitely wait for Probe Packets, which could have been lost during the journey, with the consequent delay in a successive loop recovery procedure.’’

3.4. MQ-Routing algorithm detailed summary Tables 1 and 2 summarize the packet types used by the protocol, and the actions taken by the algorithm upon occurrences of the possible events, respectively.

3.5. MQ-Routing properties At first sight, the MQ-Routing approach seems to be very similar to the common distance vector approaches, but it presents some key differences. In each node, distance vector protocols associate to each destination a single value, representing the distance to reach a determined destination via the selected next-hop, and only one path a time is available and exploited. The consequence is that the algorithms are not effective in switching to a different next-hop (handover), and also in balancing the traffic with respect to a specific metric. Therefore, distance vector approaches are not properly adequate for mobile and energy-constrained scenarios.

The MQ-Routing strategy, on the contrary, maintains for each destination a vector of values, each one associated to a next-hop node. This helps in performing fast handovers and allows exploiting different paths as network conditions vary. In fact, the best paths vary over time according to the node mobility, to the node discharging occurrences and to the time-varying discount factors. In conclusion, the MQ-Routing protocol has fault-tolerant characteristics: if a link failure occurs, a new path is immediately set up, since every node keeps estimates of the conditions of all the available paths; moreover, every node tries to predict when a link failure occurs (both via the link stability metrics, which account for the node mobility, and via the energy metric, which accounts for the node discharging) in order to trigger path changes even before a link fault occurs. In most of the fault-tolerant routing algorithms reported in [22], the fault tolerance is achieved by means of redundancy of information, i.e., the same data is replicated onto multiple paths linking source and destination nodes. These fault-tolerant approaches are then not suitable for critical scenarios, where the battery duration is one of the crucial constraints; in fact, sending the same information on multiple paths leads to the waste of precious resources. Differently, MQ-Routing utilizes only one path at a time and rapidly switches to the best one, according to the Q-values in the routing table. 4. Simulations The MQ-Routing protocol was implemented in C and simulated with the OPNET 16.0 [34] environment, which is a widely used packet-level network simulator. Proof of

Table 1 Exchanged packets description. Packet name

Description

Data Packet

Data packet, with MQ-Routing control 1. destination_node: traffic flow destination node; overhead piggy-backed. 2. next_hop: next-hop node toward the destination, as decided by the MQ-Routing algorithm; 3. Sp: sequence number to detect possible loop formation.

Main packet fields

Hello Packet

Control packet, periodically broadcast 1. qi_values: list of Qi(d, n) values for every destination d e V and for every neighbour n e Ni(t); by the generic node i to its 2. residual_energy: node i residual energy (see Section 3.1.2); neighbourhood. 3. position: node i current position (see Section 3.1.1); 4. speed: node i current speed (see Section 3.1.1); 5. direction: node i current direction (see Section 3.1.1); 6. mobility_factor: node i mobility factor (see Section 3.1.1).

Probe Packet

Control packet aimed at confirming a 1. destination_node: the node which caused the loop; 2. originating_node: the node which detected the loop; loop 3. visited_nodes: the list of nodes involved in the loop already visited by the Probe Packet; detection (see Section 3.3). 4. next_hop: the next hop node to receive the Probe Packet.

Path Reset Packet

Control packet aimed at avoiding the possibility to run through an already detected loop (see Section 3.3)

Path Error Packet

1. destination_node: the destination node for which updated routing information are Control packet broadcasted by the enclosed in the Path Error Packet; generic node i as soon as a link failure event is 2. new_next_hop: the re-calculated node which allows to reach the destination_node from node i; generated (see Table 2) 3. qi = Qi(destination_node, new_next_hop) inside node i and containing the new actionvalue related to destination_node and the new_next_hop.

1. originating_node: the node which detected the loop; 2. visited_nodes: the list of nodes involved in the loop. 3. next_hop: the next hop to which forward the Path Reset Packet.

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

9

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx Table 2 Description of events and subsequent actions. Event

Actions undertaken by the generic node i

Initialization

Node i actions:  initialize the neighbour set Ni = £.

Hello transmit timeout

Event triggered each THELLO seconds Node i actions:  broadcast a Hello Packet, with field set as specified in Table 1.

Neighbour hold timeout

Event triggered to keep the node i neighbour set Ni up to date. If the timeout expires in node i for neighbour j, node i actions are as follows:  update the neighbour set N i N i n fjg;  update the action-value function Qi(d, j) = 0, " d e V.

Loop detected

Event triggered when a loop involving the node i is detected through the analysis of the sequence number Sp, piggybacked to the Data Packets. Node i actions:  create a Probe Packet p with p.originating_node = i, p.destination_node = d and p.visited_nodes = {s};  choose next-hop k as k ¼ arg max Q i ðd; lÞ: l2N i ðtÞ

if k is not available, then discard p; otherwise set p.next_hop = k, start the Loop Recovery timer and forward p to neighbour k. Link failure detected

Event triggered when no Hello Packets are received by node i from a node j within the interval defined by the Neighbour hold timeout (i.e., when the neighbour j is not connected to the node i anymore). Node i actions:  update the neighbour set N i N i n fjg;  update the action-value function Qi(d, j) = 0, " d e V;  create a Path Error Packet p: choose next hop k computed as k ¼ arg max Q i ðd; lÞ; l2Ni ðtÞ



if k is not available, set p.next_hop = null; otherwise, set p.next_hop = k and qi = Qi(d, k); broadcast p.

Hello Packet received from node j

Node i actions:  update the discount factor cij(t) (Eqs. (3.2)–(3.6)) based on the Hello Packet fields;  update the neighbour-set and the action value function: if j R Ni, then Ni Ni [ {j}; update Qi(d, j), " d e V, according to Eq. (3.1).

Data Packet p received from node j

Node i actions:  if p.destination_node = i, the packet is delivered;  otherwise, choose next-hop node k as k ¼ arg max Q i ðd; lÞ: l2N i ðtÞnfjg

if k is not available discard p; otherwise, transmit packet p to the next hop k. Probe Packet p received

Node i actions:  if p.originating_node = i (a loop is detected), create a Path reset message q, with q.destination_node = p.destination_node, q.originating_node = i, q.visited_nodes = p.visited_nodes; forward q following the reverse loop path defined by q.visited_nodes;  otherwise update the list of visited nodes p.visited_nodes {i, p.visited_nodes}; choose k ¼ arg max Q i ðd; lÞ; l2Ni ðtÞ

if k is not available, p is discarded; otherwise: set p.next_hop = k; forward p, to the next-hop k. Path Reset Packet p received from neighbour j

Node i actions:  if p.originating_node = i, terminate loop recovery procedure with success; stop the Loop Recovery timer;  otherwise set Qi(p.destination_node, j) = 0; retrieve the next-hop k from the reverse path contained in p.visited_nodes; set Qi(p.destination_node, k) = 0; set p.next_hop = k; forward p to node k.

Path Error Packet p received from node j

Node i actions:  if p.destination_node – i, if p.qi > 0 (p.destination_node is reachable through j), update Qi(p.destination_node, j) using the value p.qi (see update rule 3.1); (continued on next page)

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

10

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

Table 2 (continued) Event

Actions undertaken by the generic node i otherwise, set Qi(p.destination_node, j) = 0; compute k ¼ arg max Q i ðp:destination node; lÞ; l2N i ðtÞ

set Qi(p.destination_node, j) = 0; 0 compute k ¼ arg max Q i ðp:destination node; lÞ; l2N i ðtÞ

if k0 is available and k – k0 , create a Path Error Packet q, setting q.next_hop = k0 and q.qi = Qi(p.destination_node, k0 ); else if k0 is not available and k – k0 , create a Path Error Packet q, setting q.next_hop = null;  else stop. Loop Recovery timeout

Node i actions:  unsuccessfully terminate loop recovery procedure. The loop recovery procedure could be re-started in case of a new loop detection (see Loop Detected event).

concepts simulations, taking into consideration the different proposed metrics, are reported in Sections 4.1 and 4.2, where the MQ-Routing is compared to the Q-Routing and to the standard OLSR [10] protocol. All simulation parameters are in line with the actual civil protection requirements collected within the MONET project [35]. Tables 3–5 define the mobility model (namely, the Random Waypoint mobility model [36]) and

Table 3 Mobility model. Attribute

Value

Type x_min (m) y_min (m) x_max (m) y_max (m) Speed (m/s) Pause time (s) Start time (s) Stop time (s)

Random waypoint 0 0 Scenario dependent Scenario dependent Scenario dependent Constant(5) Constant(30) End of simulation

Table 4 OLSR parameters (see [10] for explanations). Attribute

Value

Willingness Hello interval (s) – Topology control interval (s) Neighbour hold time (s) Topology hold time (s) Duplicate message hold time (s)

Default (3) 2–5 6 15 30

Table 5 Q-Routing/MQ-Routing parameters. Attribute MAX

c a

THELLO (s) Neighbour hold timeout interval (s) Loop Recovery timeout (s)

Value 0.95 0.95 Scenario dependent Scenario dependent 0.2

the routing protocols parameters (both for OLSR and for Q/MQ-Routing) used in the simulations. The wireless link model used in all the simulations is the Wi-Fi 802.11a [37] at 54 Mbps. The transmission range TX is 115 m. TCP and UDP protocols were considered in generating the node traffic. For the sake of simplicity, the energy model is linear with the amount of bit to be transmitted, is independent of the distance of the peer node and neglects the energy spent in packet processing. The consumption due to data processing is a negligible contribution compared to the consumption due to the transmission of data and all the activities connected to the usage of radio equipment (see for instance [38] or [39]). The amount of energy spent for processing increases in case of complex and computationally expensive procedures, which is not the case of the simple consultation of routing tables and/or periodic exchange of packets considered in the paper in question. Moreover, the authors assumed that the energy spent for processing is similar for both the tested routing algorithms (MQ-Routing and OLSR) because (i) both the algorithms analyze the packets and read the routing tables, (ii) MQ-Routing calculates the simple update rule (defined in Eq. (3.1)) while OLSR requires to explicitly compute a solution to the shortest path problem (therefore, in the comparison, OLSR takes advantage from the neglecting of the energy spent for processing). The energy model does not consider the distance of the peer node since the experimental equipment utilized in the MONET [1] project (as numerous ad hoc network nodes) do not allow to modulate, in a precise way, the transmission power in function of the distance between the transmitter and receiver nodes. However, the authors believe that the performance of the algorithm would be similar even in a network in which the transmission energy is modulated on the transmission distance. In particular, the worst case would be the one in which the MQ-Routing algorithm decides to choose the most charged path, among the available ones in the network, and this path is composed by nodes disposed very far one from each other (obviously, within the limit of nodes coverage): these nodes will rapidly discharge, because they are transmitting at full power. The algorithm would benefit from the

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

knowledge of the energy model: the routing decision could be based on the predicted energy consumption of the path (just as with the distance metric). In this respect, we regard this comment as a useful suggestion for our future work. Nevertheless, even in the case detailed above, the current MQ-Routing algorithm would avoid a rapid discharge of the nodes belonging to the path by deciding a path handover, thanks to the continuous update of the available residual energy of the nodes. Hence, the use of such a simplified model is justified by the fact that the routing approach is marginally affected by the chosen energy model, since it mainly needs to be aware of nodes residual energy, regardless of the causes of the battery discharges. Protocol performances are evaluated in terms of throughput and minimum lifetime, defined as the period of time before the first node fully drains its energy. 4.1. Comparison with Q-Routing In the simulations described in this section, the MQRouting is compared to the Q-Routing in a fixed topology scenario, where the Q-Routing converges to the shortest path. In this respect, the comparison results are also meaningful considering other shortest path algorithms. MOB As nodes are fixed, the factors cGPS ðtÞ are ij ðtÞ and cj identically set equal to 1, so that the discount factor (3.2) only depends on the energy metric: cij ðtÞ ¼ cMAX  cEN j ðtÞ. Two scenarios were identified and simulated in order to test the energetic load balancing functionalities of the protocol: a simple topology with a single flow scenario, useful to explain the different behaviour of the protocols, and a scenario encompassing a grid topology, with a variable number of nodes, useful to analyze the scalability properties. The convergence time of the MQ-Routing depends on the distance among the source and destination nodes. The duration of all simulations reported in this section is set to 360 s. Such duration is appropriate to show the algorithm convergence in the simulated MANET networks, which, as described in the MONET project [1], consist of about 10–20 nodes. Finally, the simulation results reported in this section are the average of the results of 15 simulation runs, each one performed with a different random seed. 4.1.1. Single flow scenario The topology of the single flow scenario is showed by Fig. 2, where the percentages close to the nodes represent the node residual energy at the beginning of the simulation. The Source node can be seen as a gateway for the communication with the external world (for example a hybrid satellite/MANET node as defined in MONET project) and the traffic flow could represent an audio call coming from a specific external control center to a rescue team member. In this scenario, a proper energy balancing is desirable to increase and balance the nodes lifetime. The single flow starts at Source node and terminates at Destination node, and it is characterized by a constant bitrate of 100 kbps. In the simulation, it is assumed that a node with a fully charged battery is able to transmit 20,000 packets. The THELLO is set to 1 s (one Hello Packet

11

Fig. 2. Single flow scenario topology (the dashed lines are wireless links).

per second). From the figure, it is evident that two distinct and disjoint paths are available. The learning rate is set to a high value (a = 95%) to guarantee rapidly adaptation to network changes. As previously mentioned, the proposed algorithm is compared with the Q-Routing, representing a distributed shortest path algorithm. Fig. 3 shows the throughput of the two approaches. Although from Fig. 3 it might appear that the results of the two algorithms is the same (the flow duration is the same due to the topology characteristics), the actual behaviours of the algorithms are different. Firstly, note that the energy-aware MQ-Routing algorithm prevents the sudden decrease of transmission rate which occurs when the Q-Routing algorithm performs exploration to find the new path, at about 150 s, as the first path becomes unavailable due to discharged nodes. Moreover, Fig. 4 shows how flow packets are forwarded through the network according to the two algorithms. The shortest path algorithm exploits the (shortest) path Source-0-1-4-Destination until the half-charged nodes 1 and 4 become discharged; afterwards, it starts using the second path Source-0-2-3-5-6-Destination until the halfcharged nodes 2, 3, 5 and 6 become uncharged and the flow cannot be forwarded anymore to its destination. The MQ-Routing, instead, already after about 40 s starts to exploit alternatively the two paths, trying to keep the residual energy in the nodes balanced. Fig. 5 shows the residual energy decreases of nodes 1 and 2, which are representative nodes of the two paths. The Q-Routing algorithm, as above-mentioned, exploits one path a time, leading to a rapid discharge of node 1, before starting to use the second path via node 2. In this way, node 1 is discharged after about 150 s. The MQ-Routing, by using simultaneously the two paths, leads to a slower and homogeneous discharge. In fact both the nodes are discharged at about 250 s. In this simple scenario, the minimum node lifetime has been increased of about 65%. A drawback of the proposed solution is that the delay is more varying compared to the Q-Routing, as shown in Fig. 6. Furthermore, in order to show the feasibility of the MQRouting, the routing overhead in the single flow scenario is reported in Fig. 7. MQ-Routing is compared with the standard OLSR protocol: the routing overhead is expressed in

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

12

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

Fig. 4. Single flow scenario traffic forwarded comparison: (a) Q-Routing and (b) MQ-Routing. Fig. 3. Single flow scenario throughput comparison: (a) Q-Routing and (b) MQ-Routing.

percentage, as the fraction between the network overhead and the application throughput. After the initial transitory phase, both the routing protocols stabilize. The average percentage values are shown in Table 6. Also in view of the advantages of the MQ-Routing in terms of energy balancing (in the simulation, the energy consumed by the traffic overhead is taken into account) and scalability (see the following sections), the 1% overhead increase appears to be absolutely tolerable.

By analyzing the curves in Fig. 9, it turns out that the minimum lifetime of the proposed energy-aware MQRouting increases linearly with the number of nodes n, whereas the Q-Routing algorithm produces an about constant minimum lifetime. This result depends on the fact that the shortest path algorithms simply choose one path a time, causing the discharge of the shortest path nodes independently of n. On the contrary, the proposed energy-aware MQ-Routing simultaneously uses all the possible paths, whose number increases with n. 4.2. Comparison with OLSR

4.1.2. Large scale scenario This section shows the difference in minimum node lifetime and the overall throughput in a grid scenario in function of the number of nodes n e {4, 9, 16, 25, 36, 49, 64, 81, 100}. The scenario is shown by Fig. 8 for n = 16. In each simulation, four flows start from the four edges and are directed to the opposite edges. All node batteries are full at simulation start time.

This section reports a set of simulations aimed at validating (i) the link stability metrics and (ii) the whole set of proposed metrics, also in terms of scalability as the area size grows. Then, four simulation sets was carried out: 1. GPS-aware link availability metric scenario simulation. 2. Throughput increase using link stability metrics.

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

13

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

14

Routing overhead [%]

12 10 8 6 4 2 0 0

100

200

300

400

Simulation time [s]

MQ-Routing OLSR Fig. 7. Routing overhead comparison in single flow scenario.

Table 6 Routing overhead comparison (%). Routing protocol

Average routing overhead (%)

OLSR MQ-Routing

3.49 4.51

Fig. 5. Single flow scenario residual energy comparison: a) Q-Routing and (b) MQ-Routing.

0.0009 0.0008

End to end delay [s]

0.0007 0.0006 0.0005 0.0004 0.0003 0.0002 0.0001

Fig. 8. 4  4 Grid topology.

0 0

50

100

150

200

250

300

Simulation Time [s] Energy-aware MQ-Routing Simple QRouting

Fig. 6. End-to-end delay comparison.

3. Throughput and lifetime results considering all metrics together. 4. Scalability comparison in terms of area size growth. In these simulations, the THELLO and Neighbour hold timeout interval parameters of MQ-Routing are set equal to the

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

14

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

Fig. 10. GPS-aware example scenario; the arrow indicates the direction of node 2, which is leaving the network.

and the second one via nodes 1-3-4-5-6. Node 2 starts to move from its initial position at time 120 s and reaches the transmission range limit of 115 m after 190 s, moving with a constant speed of 2 m/s. The simulation results reported in this section are the average of the results of 15 simulation runs, each one performed with a different random seed. Fig. 11 shows the traffic received (at application level) at destination node, comparing OLSR with the MQ-Routing. OLSR needs to wait for node 2 to reach the transmission limit TX to switch the path, and this happens only after a prefixed Neighbour Hold Time timeout expires, starting from the last Hello Packet received by node 1. The GPSaware MQ-Routing, in contrast, starts switching the path in advance, estimating that the link will soon go down: in this way, the throughput of the received traffic does not suffer from the switching and from the consequent loss of packets (the packet loss in OLSR is about 73.6% in the 6 s needed for the switching procedure). Fig. 9. Comparison with the increase of the number of network nodes in a grid topology: (a) minimum node lifetime and (b) average traffic flow duration.

Hello Interval and the Neighbour hold timeout parameters of OLSR (see Table 4 for details).

4.2.1. GPS-aware link availability metric scenario simulation This section reports a network scenario, shown in Fig. 10, in which two alternative paths are present; due to mobility, one of the feasible paths becomes unavailable. Also in this scenario, the Source node could represent a hybrid gateway, connecting the MANET with the external world (e.g. one or more control centers). A bidirectional Pulse Code Modulation (PCM) constant audio flow at 64 kbps is set up between the Source and Destination nodes. The factors cEN j ðtÞ and cMOB ðtÞ are identically set equal to 1, so that the discount j factor (3.2) only depends on the GPS-aware link availability metric: cij ðtÞ ¼ cMAX  cGPS ij ðtÞ: There exist two different paths between Source and Destination nodes, the shortest one via nodes 1 and 2

Fig. 11. Traffic received comparison using GPS-aware link availability metric.

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

Note that the Q-Routing suffers from the same drawbacks of OLSR, since both depend uniquely on the hello intervals and do not estimate link availability time. In this respect, the enhancement provide by the MQ-Routing is due to the new metric and to the introduction of AODVlike path error messages. 4.2.2. Throughput increase using link stability metrics This section reports the behaviour of the MQ-Routing when the node speed increases. Since the residual energy is not the focus of this simulation, the factors cEN ij ðtÞ is identically set equal to 1, so that the discount factor (3.2) only depends on the link stability metrics: cij ðtÞ ¼ MOB cGPS ðtÞ. ij ðtÞ  cj The simulated scenario is a square area of 400 m  400 m, in which 20 nodes are deployed and move following a Random Waypoint mobility model [36] with a prefixed speed. The learning rate parameter a of the MQRouting approach is set to 0.99, very close to 1, in order to rapidly adapt to network changes. Four bidirectional GSM G.711 audio flows at 13.6 kbps are set up between four network nodes. The simulations have a duration of 300 s and were performed by varying the speed in the interval 1–15 m/s (as proposed by MONET project in [35]). For each speed, the simulation was repeated 50 times. The application level normalized throughput (expressed as the percentage of traffic received at the application layer in destination node) was analyzed and the behaviour is reported in Fig. 12, where is also compared to OLSR. The figure shows that the link stability metrics consistently improve the algorithm behaviour at high speed. The throughput increase of the MQ-Routing compared to

Fig. 12. Throughput comparison using link stability metrics.

15

OLSR averaged along 1–15 m/s speeds is 10.2% and reaches a peak of 25.9% at 15 m/s. This result is intuitive because the link availability and the mobility metrics are more effective at high speeds, as the estimated node positions rapidly vary and the mobility factor assumes more importance. 4.2.3. Throughput and lifetime results considering all metrics together This section reports the normalized throughput and the minimum lifetime (expressed in percentage on the single simulation time) in case all metrics are considered together, as in Eq. (3.2). The simulation scenario is a square area of 400 m  400 m, in which 20 nodes are deployed and move following a Random Waypoint mobility model with a prefixed speed. The learning rate parameter of the MQ-Routing approach is set to 0.99, very close to 1, in order to rapidly adapt to network changes. The node battery allows transmitting about 10 MB. Four bidirectional video flows at 512 kbps (as proposed by MONET project in [35] for video

Fig. 13. Throughput and minimum lifetime comparison using all metrics.

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

16

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

service requirements) runs between 4 network nodes. The simulations, with a duration of 300 s each, were performed with different speed in the interval 1–15 m/s and, for each speed, the simulations were repeated 50 times. Fig. 13 shows the average application level normalized throughput and the average minimum node lifetime, expressed in percentage on the application level traffic sent and the maximum battery availability, respectively: The throughput of the MQ-Routing is about the same of OLSR (only at low speeds it is effectively increased, thanks to the energy metric which better distributes the traffic load) but the minimum lifetime is considerably increased, especially at low speeds. This fact is a valuable result especially for disaster relief scenarios, because such a low speed interval contains the human speed. The throughput increase of the MQ-Routing compared to OLSR, averaged along 1–15 m/s speeds, is of 1.74% and reaches a peak of 6.95% at speed 2 m/s. The minimum lifetime increase of the MQ-Routing compared to OLSR, averaged along 1–

15 m/s speeds, is of 6.79% and reaches a peak of 16.49% at speed 1 m/s. 4.2.4. Scalability comparison in terms of area size growth Scalability is an important quality of any routing protocol in a network, because is related to the ability to adapt in a performing way to the increase of specified parameters of interest. In this section, we focus on the behaviour of the proposed MQ-Routing in function of the area of the network, comparing the results with OLSR. A set of simulations was carried out, with fixed node speed of 2 m/s, and considering a node battery able to send 24 MB of traffic (including both application and routing overhead traffic). Eight video flows at 512 kbps were executed through the network composed by 20 nodes which move following a Random Waypoint model. Each simulation lasts 600 s, and for each area size 50 simulations were performed. The simulation results are summarized in Fig. 14. The comparison term in the second figure has been chosen to be the product between the normalized throughput and the minimum lifetime in percentage because the two comparison elements are in contrast each other: in fact, the increase of throughput produced by a better adaptability to network changes, thanks to the link stability metrics, can lead to a more rapid discharge of batteries. By analyzing the figure, it is evident that, compared to OLSR, the MQRouting more efficiently adapts to the increase of network area. This fact can be justified by the capability of the algorithm to predict the behaviour of the nodes by means both of an analysis of the future movement through GPS information and of the choice of the most ‘‘stable’’ nodes. The average throughput increase compared to OLSR, calculated through all the network sizes, is of about 13% and the average product between normalized throughput and minimum node lifetime increase is of about 9.5%. The peaks values of the increases are reached both at 64 ha of network area size and respectively amount to 22.3% and 17.3%. 5. Conclusions and future work

Fig. 14. Comparison with OLSR when the network area increases.

This paper proposes a proactive version of the Q-Routing [3] algorithm, based on Reinforcement Learning, suitable for Mobile-Ad-Hoc-Networks (MANETs), in particular with the aim to be aware of node residual energy and mobility. To this purpose, specific combined metrics are reported and introduced. Simulation showed that the minimum node lifetime is increased, leading to a fairer energy balancing, and that the application level throughput is increased. The algorithm was compared with the original Q-Routing and with the Optimized Link-State Routing (OLSR) Protocol, in scenarios with characteristics (in terms of application flows, concurrent users, intervention areas and node speed) in line with disaster relief scenarios requirements, as proposed in the MONET FP7-ICT project [1]. The proposed approach suggests a number of interesting directions for further work. For instance, an energy model including the distance of peer nodes or the packet

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

processing could be considered. Furthermore, the algorithm formulation allows seamless introduction of further metrics. An example could be the introduction of collisions-aware metrics, since reducing collisions is of crucial importance to increase the global network throughput, especially in standard 802.11 CSMA/CA Wi-Fi links [37]. References [1] MONET (Mechanisms for Optimization of hybrid ad-hoc networks and satellite NETworks), FP7-ICT project (Contract no.: 247176). . [2] J.A. Boyan, M.L. Littman, Packet routing in dynamically changing networks: A reinforcement learning approach, Advances in Neural Information Processing Systems 6 (1994) 671–678. [3] R.S. Sutton, A.G. Barto, Reinforcement learning – an introduction, 1998. [4] F. Delli Priscoli, A. Pietrabissa, Hop-by-hop congestion control for adhoc wireless-LAN: a model-based control approach, International Journal of Control 78 (12) (2005) 887–905, http://dx.doi.org/ 10.1080/00207170500199718. [5] D. Macone, G. Oddi, A. Palo, V. Suraci, A dynamic load balancing algorithm for quality of service and mobility management in next generation home networks, Telecommunication Systems Journal (Springer), 2013, in press. [6] A. Pietrabissa, An alternative LP formulation of the admission control problem in multi-class networks, I EEE Transaction on Automatic Control 53 (3) (2008) 839–845, http://dx.doi.org/10.1109/ TAC.2008.919516. [7] S. Corson, J. Macker, Mobile Ad hoc Networking (MANET): Routing Protocol Performance Issues and Evaluation Considerations, IETF RFC2501, January 1999. [8] L. Chen, W.B. Heinzelman, A survey of routing protocols that support QoS in mobile ad hoc networks, Network, IEEE 21 (6) (2007) 30–38. [9] H. Badis, K. Al Agha, Quality of Service for Ad hoc Optimized Link State Routing Protocol (QOLSR), draft-badis-manet-qolsr-04.txt, October 2006. [10] Optimized Link State Routing Protocol (OLSR), RFC3626. (consulted May 2011). [11] Ad hoc On-Demand Distance Vector (AODV) Routing. (consulted May 2011). [12] C.E. Perkins, P. Bhagwat, Highly dynamic Destination-Sequenced Distance-Vector routing (DSDV) for mobile computers, in: Proceedings of the Conference on Communications Architectures, Protocols and Applications (SIGCOMM ‘94), ACM, New York, USA. [13] J. Liu, Y. Lu, Energy savings in privacy-preserving computation offloading with protection by homomorphic encryption, in: Proceedings of the 2010 International Conference on Power Aware Computing and Systems, HotPower’10, Article No. 1–7. [14] Y. Xu, J. Heidemann, D. Estrin, Adaptive Energy-Conserving Routing for Multihop Ad hoc Networks, Technical, Report TR-2000-527, 2000. [15] Y. Xu, J. Heidemann, D. Estrin, Geography-informed energy conservation for ad hoc routing, in: Proceedings of the 7th Annual ACM Mobicom, 2001. [16] Y. Xiang, Y. Chai, Y. Wu, A distributed topology control algorithm based on energy awareness for heterogeneous wireless sensor networks, Journal of Computational Information Systems 6 (10) (2010) 3293–3301. [17] A. Konstantinidis, K. Yang, H. Chen, Q. Zhang, Energy-aware topology control for wireless sensor networks using memetic algorithms, Computer Communications 30 (14-15) (2007) 2753–2764. ISSN 0140-3664. [18] M. Bahramgiri, M. Hajiaghayi, V. Mirrokni, Fault-tolerant and 3dimensional distributed topology control algorithms in wireless multi-hop networks, Wireless Networks 12 (2) (2006) 179–188. doi:10.1007/s11276-005-5265-z. [19] A. Jeng, R. Jan, Adaptive topology control for mobile ad hoc networks, IEEE Transactions on Parallel and Distributed Systems, December, 2011, pp. 1953–1960. [20] A. Benslimane, R. El Khoury, R. El Azouzi, S. Pierre, Energy poweraware routing in OLSR protocol, in: Proceedings of the First Mobile Computing and Wireless Communication International Conference (MCWC), 2006. [21] X. Jing, M.J. Lee, Energy-aware algorithms for AODV in Ad hoc networks, in: Proceedings of Mobile Computing and Ubiquitous Networking, Yokosuka, Japan, January 2004, pp. 466–468.

17

[22] J. Tsai, T. Moors, A review of multipath routing protocols: from wireless ad hoc to mesh networks, in: Proceedings of ACoRN Early Career Researcher Workshop on Wireless Multihop Networking, 2006. [23] Z. Che-Aron, W.F.M. Al-Khateeb, F. Anwar, ENFAT-AODV: The faulttolerant routing protocol for high failure rate Wireless Sensor Networks, in: 2nd International Conference on Future Computer and Communication (ICFCC), 21–24 May 2010. [24] M. Cardei, S. Yang, J. Wu, Fault-tolerant topology control for heterogeneous wireless sensor networks, in: Proceedings of the IEEE International Conference on MASS, 2007, pp. 1–9. [25] C. Wu, K. Kumekawa, T. Kato, A MANET protocol considering link stability and bandwidth efficiency, ICUMT2009, 12th–14th October 2009, St. Petersburg, Russia. [26] A. Esfahani, M. Analoui, Widest K-shortest paths Q-routing: a new QoS routing algorithm in telecommunication networks, in: International Conference on Computer Science and, Software Engineering, 2008. [27] P. Wang, T. Wang, Adaptive routing for sensor networks using reinforcement learning, in: Sixth IEEE International Conference on Computer and Information Technology (CIT06), 2006. [28] P. Marbach, O. Mihatsch, M. Schulte, J.N. Tsitsiklis, Reinforcement learning for call admission control and routing in integrated service networks, Advances in Neural Information Processing Systems 10 (1998). [29] T. Tao, S. Tagashira, S. Fujita, LQ-routing protocol for mobile ad-hoc networks, Fourth Annual ACIS International Conference on Computer and Information Science (ICIS05), 2005. [30] M.L. Puterman, Markov Decision Processes, Jhon Wiley & Sons, New Jersey, 1994. [31] W. Su, S-J. Lee, M. Gerla, Mobility prediction and routing in ad hoc wireless networks, International Journal of Network management 11 (2000) 3–30. [32] Z. Jin, R.M. Murray, Multi-hop relay protocols for fast consensus seeking, in: IEEE Conference on Decision and Control, San Diego, 2006, pp. 1001–1006. [33] S. Manfredi, A theoretical analysis of multi-hop consensus algorithms for wireless networks: trade off, among reliability, responsiveness and delay tolerance, Ad Hoc Networks (2011), http://dx.doi.org/10.1016/j.adhoc.2011.05.005. [34] ‘‘OPNET Modeler 16.0’’. (consulted May 2011). [35] Deliverable 2.3 – MONET Requirements, MONET FP7-ICT Project. . [36] T. Camp, J. Boleng, V. Davies, A survey of mobility models for ad hoc network research, Wireless Communication & Mobile Computing (WCMC): Special issue on Mobile Ad Hoc Networking: Research, Trends and Applications 2 (5) (2002) 483–502. [37] IEEE 802.11™ Wireless Local Area Networks: the Working Group for WLAN Standards’’. (accessed October 2011). [38] F. Rango, J. Cano, M. Fotino, C. Calafate, P. Manzoni, S. Marano, Olsr vs dsr: a comparative analysis of proactive and reactive mechanisms from an energetic point of view in wireless ad hoc networks, Comput. Commun 31 (16) (2008) 3843–3854. [39] T. Ramrekha, V. Talooki, J. Rodriguez, C. Politis, Energy efficient and scalable routing protocol for extreme emergency ad hoc communications, Mobile Networks and Applications 17 (2012) 312–324, http://dx.doi.org/10.1007/s11036-011-0336-0.

Donato Macone was born on November 24, 1984, in Gaeta (LT), Italy. He graduated in Computer Engineering with Bachelor degree with 110/110 cum laude in February 2007 at the University of Rome ‘‘Sapienza’’ and obtained the Master Degree in Computer Engineering with 110/110 in March 2009 at the University of Rome ‘‘Sapienza’’. In November 2009 he became Ph.D student in Telecommunication Engineering in the Department of Information and Telecommunication Technologies at the University of Rome ‘‘Sapienza’’. In 2009 he joined MICIE, a STREP project within the FP7 EU ICT-SEC research programme. From year 2010 he joined TASS, an IP project within FP7 EU SEC research programme. Main competences of Donato Macone are in the field of network resource management, game theory, routing and load balancing strategies and algorithms.

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008

18

D. Macone et al. / Ad Hoc Networks xxx (2012) xxx–xxx

Guido Oddi was born on June 17, 1984, in Milan, Italy. He graduated in Computer Engineering with Bachelor degree with 110/110 cum laude in December 2006 at the University of Rome ‘‘Sapienza’’ and obtained the Master Degree in Computer Engineering with 110/110 cum laude in March 2009 at the University of Rome ‘‘Sapienza’’. In November 2009 he became Ph.D student in Systems Engineering in the Department of Computer and System Science at the University of Rome ‘‘Sapienza’’. Since 2008, he worked within the EU FP7 ICT IP project OMEGA. In 2010 he joined MONET, a STREP project within the FP7 EU ICT research programme. Main competences of Guido Oddi are in the field of network resource management, game theory, routing and load balancing strategies, C/C++/Java programming languages and Linux-based operative systems.

Antonio Pietrabissa: He graduated in Electronic Engineering with Master degree with 110/110 in 2000 at the University of Rome ‘‘Sapienza’’ and obtained the PhD in Systems Engineering in 2004 at the Department of Computer and System Science at the same university. He is currently an assistant professor with the ‘‘Dipartimento di Informatica e Sistemistica (DIS)’’ of the University of Rome ‘‘Sapienza’’, where he holds the course ‘‘Foundations of Systems Theory’’, since 2010. His research focus is the application of system and control theory methodologies to telecommunication networks. He has been involved in several EU and ESA funded projects (GEOCAST, DOMINO2, SATIP6, EuQoS, SATSIX, OMEGA, DLC-VIT4IP, MONET) and Italian funded projects on telecommunications. He is author of more than 50 papers on these topics.

Please cite this article in press as: D. Macone et al., MQ-Routing: Mobility-, GPS- and energy-aware routing protocol in MANETs for disaster relief scenarios, Ad Hoc Netw. (2012), http://dx.doi.org/10.1016/j.adhoc.2012.09.008