Time-optimized management of IoT nodes

Ad Hoc Networks 69 (2018) 1–14

Contents lists available at ScienceDirect

Ad Hoc Networks journal homepage: www.elsevier.com/locate/adhoc

Time-optimized management of IoT nodes Kostas Kolomvatsos a,b a b

Department of Informatics and Telecommunications National and Kapodistrian University of Athens, 15784, Athens, Greece Department of Computer Science University of Thessaly, 35100, Lamia, Greece

a r t i c l e

i n f o

Article history: Received 17 January 2017 Revised 22 September 2017 Accepted 17 October 2017 Available online 23 October 2017 Keywords: Internet of Things Updates management Optimal stopping theory

a b s t r a c t The vision of Internet of Things (IoT) aims to offer a vast infrastructure of numerous interconnected devices usually called IoT nodes. The infrastructure consists of the basis of pervasive computing applications. Applications can be built with the participation of the IoT nodes that interact in very dynamic environments. In this setting, one can identify the need for applying updates in the software/firmware of the autonomous nodes. Updates may include software extensions and patches significant for the efficient functioning of the IoT nodes. Legacy methodologies involve centralized models where complex algorithms and protocols are adopted for the distribution of the updates to the nodes. This paper proposes a distributed approach where each node is responsible to initiate and conclude the update process. We envision that each node monitors specific performance metrics (related to the node itself and/or the network) and based on a time-optimized scheme identifies the appropriate time to perform the update process.We propose the adoption of a finite horizon optimal stopping scheme. Our stopping model originates in the Optimal Stopping Theory (OST) and takes into account multiple performance metrics. The aim is to have the nodes capable of identifying when their performance and the performance of the network are of high quality. In that time, nodes could be able to efficiently conclude the update process. We provide a set of formulations and the analysis of our problem. Extensive experiments and a comparison assessment reveal the advantages of the proposed solution. © 2017 Elsevier B.V. All rights reserved.

1. Introduction The evolution of the Internet of Things (IoT) and pervasive computing sets new challenges in the research community. The IoT has evolved through the combination of wireless technologies (e.g., Wireless Sensors networks - WSNs) and Internet. Pervasive computing envisions the adoption of numerous devices into everyday objects. The basic concept of adopting many autonomous devices is the pervasive presence around users of various wireless technologies such as sensors, actuators, mobile phones and so on [1]. Researchers and practitioners aim to offer innovative applications on top of this infrastructure that facilitates the transition from closed networks to the interconnected autonomous nodes. Communication protocols offer the required strategies for the exchange of the collected/processed information while intelligent interfaces can assist in the management of the observed heterogeneity. Nodes come with the software and firmware of the respective manufacturers. Additionally, they act in very dynamic environments where the requirements of users and applications change continually. There are two main issues that may positively affect

E-mail addresses: [email protected], [email protected] https://doi.org/10.1016/j.adhoc.2017.10.011 1570-8705/© 2017 Elsevier B.V. All rights reserved.

nodes’ performance: (i) the application of software updates that will extend the provided functionalities making the nodes capable of working efficiently while offering new services; (ii) the application of firmware updates that cover any identified gaps in the already installed firmware; It is imperative to apply software or firmware updates and extensions in nodes’ software during their functioning. The challenge is to apply these updates in the entire network. Once nodes are in operation, they will start receiving updates that should be applied as soon as possible. However, the application of the updates should not disturb, if possible, nodes from their initial goal. Updates must be delivered in a way that conserves the limited bandwidth and connectivity of a node and eliminates the possibility of compromising security. A lot of work has been done in the reprogramming - applying updates in WSNs [2–15]. To the best of our knowledge, the vast majority of them deal with centralized approaches. In centralized models, a server is responsible to execute an intelligent algorithm that will define the steps for delivering the updates to every node in the network. The disadvantages of the centralized architecture are: (i) the server is the main responsible for delivering the updates, thus, it should apply complex algorithms and protocols to conclude the update process; (ii) the server cannot be aware of the nodes being connected in the network, thus, it can-

2

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

not be aware if all nodes have applied the updates; (iii) when the server distributes the updates, nodes should interrupt their processing tasks to apply them; (iv) the network is flooded by update distribution messages limiting the available bandwidth and reducing the network performance. In past research efforts, incremental techniques, compression, diffusion or various dissemination protocols undertake the responsibility of finding the appropriate way to deliver the updates. However, these techniques exhibit a number of drawbacks. Incremental models require an efficient management mechanism to maintain the updates history while compression and, accordingly, decompression require the adoption of complex algorithms. The additional overhead for the efficient management of incremental and compression/decompression schemes becomes significant limiting the central system’s and nodes’ performance. In addition, incremental techniques should incorporate mechanisms for handling the heterogeneity of nodes as the entire network shares the same update. Diffusion mechanisms require a large number of messages especially in dense networks, thus, limiting their performance. A large number of messages will lead nodes to spend their resources for messaging when the updates are usually distracting them from their initial goal, i.e., to perform the assigned task and deliver their results to the upper level nodes/applications. In this paper, we try to alleviate the aforementioned problems by proposing a distributed model for applying updates giving priority to the autonomous nature of nodes. In our approach, the central server acts only as a ‘repository’ of the updates. We alleviate the central server of any complexity, maximizing the performance of each node and securing the efficient application of the updates while keeping the performance of the network at high levels. Nodes have the responsibility of downloading and applying the updates when they see that it is the appropriate time to do this. This is realized through the proposed time-optimized, performanceaware mechanism defined with the adoption of the Optimal Stopping Theory (OST). The update process is initiated when the performance characteristics of the node itself and the network are of high quality to efficiently support a fast and efficient conclusion. Specifically, multiple performance metrics are taken into consideration (e.g., bandwidth, error rate, the load of each node). When a deadline is set for the updates conclusion, it defines the finite horizon version of our scheme. When no deadline is set, nodes have unlimited time to conclude the update process, however, for improving the performance, the process should be concluded as soon as possible. This scenario defines the infinite horizon version of our scheme. We focus on a finite horizon scheme as each node has a deadline to conclude the update process. The following list, reports on the benefits of the proposed solution compared to the remaining research efforts in the domain. • the server should not be aware of the specific characteristics of each node present in the network; • each node independently runs the proposed scheme and selects the time for applying the update; • in legacy systems, where the central system disseminates the updates, there is no guarantee that the nodes will be connected to the network to receive them. In our model, as nodes select autonomously the time when they will initiate the process, thus, the application of the updates is guaranteed; • it is not necessary to define and adopt complex protocols and dissemination schemes in the central system; • the proposed model is fully adapted in the performance of the network to assure the uninterrupted application of the updates. The contribution of our work is summarized as follows: • a novel time-optimized, performance-aware mechanism based on the OST, which decides when an update process, P, should be

activated. P is consisted of two stages: (i) getting the software component from the central server; (ii) installing the component into the node. • a methodology for deriving the optimal time t∗ for initiating P. At t∗ , the node enjoys the best possible performance (i.e., the highest possible network performance and the lowest possible load of the node) to support P; • our model avoids unnecessary attempts to apply P; • a comprehensive experimental analysis, the strengths of the proposed model are revealed while we compare it with other solutions. The paper is organized as follows. Section 2 reports on the related work. In Section 3, we describe and formulate our problem. Section 4 presents the analysis of the problem and provides specific formulations for its solution. Simulations reveal the performance of the proposed model in Section 5. Finally, in Section 6, we conclude our paper by presenting our future research agenda. 2. Related work Pervasive computing [16] and IoT [17] involve numerous devices interconnected to exchange information and fulfill complex tasks. This pervasive nature of the IoT vision is enhanced by the presence of sensors either in the a ‘standalone’ mode or embedded into users’ smart devices. An IoT node consists of two parts [17]: (i) the hardware that makes the device able to record / collect / send data from / to its environment, and, (ii) the software that makes the node able to process the collected / incoming data and produce some outcomes. These outcomes could be used either to be disseminated into peers; into a central system; to derive some knowledge on the node. The produced knowledge could be adopted to deliver decisions related to the presence of events and the appropriate actions towards the provision of responses to the identified events. IoT nodes come with a pre-defined software/firmware that makes them capable of completing their tasks. In this section, we report on the related work that studies schemes for delivering software updates in IoT devices. Software updates for IoT nodes are necessary after deployment to have the nodes capable of efficiently performing the assigned tasks. A survey on the adopted methodologies for updating software is presented in [18]. Firmware updates are necessary in nonmodular sensor operating systems (e.g., Tiny Operating System (TinyOS) [20]) that require the complete update of the firmware to have updated sensors. A number of mechanisms are proposed for transferring and installing firmware updates [2,5,8,9,11,12,14]. Apart from firmware, generic re-programming (i.e., software components that are not directly related to the firmware) is a similar problem that aims to extend or correct the sensors functionality during their lifetime. Re-programming is related to updates in the software components that aim to provide extensions (i.e., new functionalities) or solve possible errors [7]. To maintain the software, it is necessary to minimize the size of the transferred data [12]. The envisioned algorithms target to minimize the time required for applying any update through the minimization of the amount of data that will be transferred to nodes [21]. Incremental updates and data compression could be adopted for such purposes [22]. Sensor operating systems like Contiki [23], Lite Operating System (LiteOS) [24] and the Resilient, Extensible anf Threaded Operating System (RETOS) [25] support dynamic linking and loading, however, the adopted Executable Linkable Format (ELF) format (Contiki adopts the format) leads to binaries potentially large to be transferred in WSNs [22]. Data dissemination protocols are the key part of a dissemination strategy for transferring software updates. Some examples are discussed in [3,4,6,7,10,13,15]. In general, dissemination protocols

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

assume an ad-hoc network and use a form of controlled flooding [18]. They differ from the data collection protocols in storage, coverage and data flow. In WSNs, resources are limited and nodes cannot cache overheard packets which might not be useful [3]. A protocol running in sensor networks must be simple and easily implemented [19]. Additionally, it should not require a large number of messages otherwise, bottlenecks will be present with negative effects in the performance of the network. The normal pattern for dissemination protocols is a three-step process [18]: (i) the advertisement of available software; (ii) the selection of a source; and (iii) the reliable downloading to the target. A subscription approach could be adopted, however, this results a significantly increased overhead in the network and, more specifically, in the server, where the updates are present. Dissemination protocols are executed by the corresponding WSN middleware. The middleware could be event or application driven. In the event based middleware [26,27], the application denotes its interest in specific events and after the firing of the events updates take place. Some examples are provided: (a) in [28], where a set of event operators for describing event patterns has been proposed; (b) in [29], where a publish/subscribe scheme is proposed on top of the TinyOS. The Middleware Linking Applications and Networks (MiLAN) solution [30] is an example of an application driven middleware. In such types of middleware, applications specify their requirements through a standard Application Programming Interface (API). More recent advances involve protocols adopted in Software Defined Networks (SDNs) where virtualized resources are defined. The provided protocols disseminate the updates to the actual devices through the management of a set of virtualized resources. In [31], the authors present the OpenFlow protocol that supports packet forwarding across switches. In addition, they propose the OpenFunction which is a secure communications protocol. Some widely cited research efforts in the domain are as follows. Trickle [8] is an algorithm that disseminates and, accordingly, maintains software updates in WSNs. Trickle adopts an epidemic approach with scalable multicasting through which updates are periodically transferred to the nodes. Special attention, on the elements that could be exchanged between the nodes, is paid by the data discovery and Dissemination Protocol (DIP) [32]. The protocol tries to randomly scan the network to detect new items while maintaining a low latency. DHV [33] is an efficient code consistency maintenance protocol ensuring that nodes will, eventually, have the same code. The Multicast-based Code redistribution Protocol (MCP) [35] is another protocol that performs code maintenance in multiple nodes. MCP ‘forces’ every node to maintain a table that contains the information of known applications. The table enables sending out multicast-based code dissemination requests. Multi-hop, Over-the-Air code distribution Protocol (MOAP) [14] uses a store-and-forward approach providing a pattern of updates. The updated code is broadcasted in a neighbourper-neighbour way forcing nodes to disseminate the incoming code to reduce the latency. Deluge [2] is a protocol that builds on top of algorithms related to density-aware, epidemic maintenance protocols and includes several optimizations. It adopts Trickle for the advertisement of code and splits the code into a set of fixed-size pages. Through this approach, the time required for the propagation of a large component is reduced. Stream [11] adopts Deluge and optimizes the code parts sent in the network. Stream deals with pre-installing in each node the re-programming application. Hence, Stream transmits the minimal support (about one page) needed for the activation of the re-programming image. Resourceawareness, time-efficiency, and the integration of security solutions are adopted in the model presented in [36]. A multi-hop propagation scheme is proposed which is enhanced by security codes and means from fuzzy control theory. MELETE [37] is designed to support multiple concurrent applications in a WSN. It assumes

3

that the network consists of groups of nodes that execute different tasks. The framework adopts a group-keyed model to selectively distribute the code to interested nodes, and reactively distribute code only when it is required. In [38], the authors propose and describe an Adaptive Code Dissemination Protocol (ACDP) for reprogramming WSNs. The ACDP decreases the communication cost avoiding unnecessary calculations and transmissions. The proposed model involves a load balancing feature that is capable of extending the lifetime of the entire network. Finally, a set of research efforts deal with the management of the network reprogramming. Network coding can reduce traffic and improve the transmission performance. In this research domain, various code dissemination protocols are proposed in [39] and [40]. Fountain Reprogramming Protocol (FRP) [39] and SYNAPSE [40] adopt specific coding methodologies to minimize the traffic. In [41], the authors propose a model where the sending node can generate encoded packets from a set of original packets. The receiving node can decode successfully after receiving any number of the encoded packets. In [42], the proposed model adopts a random linear network coding RLC (Random Linear Codes) [43]. Sending nodes randomly generate a set of coefficients and calculate the linear combination of packets. 3. Problem statement 3.1. Preliminaries In the following problem description and analysis, we adopt multiple variables to depict entities/objects/concepts (e.g., nodes) participating in the envisioned setting. Subscripts in the adopted variables indicate the index of the corresponding entity/object/concept (e.g., node, metric, stage) while superscripts denote the exponents of the variables in the corresponding mathematical formulation. We envision a setting where a set of IoT   nodes N = n1 , n2 , . . . , n|N | perform specific tasks. An IoT node, ni , is a physical device embedded with electronics and software having capabilities of collecting, processing and exchanging of data. Tasks executed by nodes are independent each other (even if some of them could perform the same task). P is an independent task that should be immediately executed, if possible, without disturbing nodes from their goals. An update is a change in the nodes software that could be [22]: (i) an update of the operating system; (ii) an update of an application; (iii) an addition of a new application; (iv) a modification of parameters in an existing application. In the discussed setting, there is available an update server S that is a software component responsible to manage the updates available for the underlying nodes. S is also responsible to secure the dissemination/reception of the updates. In short, some of the S’s responsibilities are [44,45]: (i) inform nodes about the new updates; (ii) pushes the data of software updates to the nodes; (iii) create and apply security protocols to enforce the integrity of the update components; (iv) secure the reception of the update components in the entire set of nodes. In this paper, we try to alleviate the load of S when serving a high number of nodes. When a high number of nodes try to contact S, download and install updates, probably, bottlenecks could be arisen. In such cases, S should adopt an intelligent algorithm or be accompanied by a powerful hardware to efficiently serve the increased load. In general, the execution of P is characterized by bursts allocated near the time of the updates announcements. For alleviating the problem and avoid possible bottlenecks, we propose a distributed scheme that each node adopts to be able to get and apply the available updates. With our scheme, each node decides independently when to conclude P. This decision is affected by the current node performance and the state of the network. An update process, P, (from the node perspective) is concluded af-

4

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

ter the reception of the corresponding S’s message and consists of the following steps (for each ni ∈ N ): (i) ni is connected to S; (ii) ni downloads the update; (iii) ni applies the update locally. We assume that S, when an update is available, sends a lightweight message to the respective nodes for the presence of the update file / software. Based on the criticality of the update, the message could be accompanied by a deadline T. This means that till the expiration of T, each ni should conclude P. ni should conclude P immediately, however, under specific constraints (to secure a high level of performance). Constraints are related with ni ’s performance parameters that indicate the secure conclusion of P. The deadline for concluding P is defined by selecting an initial time, say t0 , and a final time, i.e., T satisfying the inequality 0 ≤ t0 ≤ T ≤ +∞. T is called the horizon of the problem. When T < +∞, we speak about a finite horizon problem and when T = +∞, we speak of an infinite horizon problem [46]. In this paper, we focus on the finite horizon version of the proposed scheme while the study on the infinite horizon version is left for future work. Assume now that a new update is available in S. We define the update era UE as the time required by nodes to apply P. UE is an interval [1, U] in which the entire set of nodes should conclude P. When U = T , we consider the finite horizon version while when U = +∞, we consider the infinite horizon version of our scheme. It should be noted that multiple UE’s could be run in parallel. Each UEi , i = 1, 2, . . . , ∞ concerns a subset of nodes. In short, the research challenges of our work are: (i) Each node should monitor its load and the network status to retrieve performance data. These data will be adopted to become the basis for the decision making of the node related to the conclusion of P. The challenge is to select the appropriate metrics to be monitored by the decision making mechanism. The monitoring of the network performance will add overhead, however, this overhead will be eliminated if the data reporting is characterized by low frequency. Recent studies show that when the reporting frequency of the monitoring data is low also the impact on the end-user traffic performance will increase [47]. The frequency of the monitoring process affects the throughput and the end-toend delay. When the reporting frequency is set to high values, i.e. 15 seconds, the impact is almost zero [47]. In this paper, we consider that nodes adopt a frequency that leads to almost zero overhead for network monitoring. An analytical study on the constraints of the network monitoring overhead is beyond the scope of this paper. (ii) Based on the collected performance data and after the reception of an update message from S, for every UE, find the optimal stopping time t ∗ ∈ [1, U] where the node will stop the monitoring process and initiate P. 3.2. Update management optimization With the proposed mechanism, we aim to offer a timeoptimized decision scheme for each ni . Such a decision mechanism tries to find the appropriate time to initiate and conclude P. As mentioned, the decision will be based on specific performance metrics. A performance metric mk , k = 1, 2, . . . measures the activities and the performance of an entity. Nodes could be affected by ‘internal’ or ‘external’ issues. For instance, a limited memory, an increased number of tasks or limited processing capabilities are considered as ‘internal’ issues while interference or a limited bandwidth are considered as ‘external’. Both types of parameters could be monitored adopting the corresponding metrics. In our model, we focus on two types of metrics related to (i) network, and (ii) nodes’ performance. Network performance metrics can be categorized into [48]: (i) availability; (ii) packet loss and error; (iii) delay;

(iv) bandwidth. P, usually, involves large amounts of data, thus, for our study, we adopt bandwidth (let us define parameter b to depict the current bandwidth). b assess the amount of data that a node can transfer through the network in a time unit and it is updated during nodes’ activities. Our model can be easily extended to include more metrics into the decision scheme. Node performance parameters are related to the node behaviour and the load concerning the executed tasks (let us define the parameter l to depict the current ‘availability’ of each node). For simplicity, we do not focus on hardware related issues (e.g., memory size, CPU speed, storage used) and assume that l depicts if a node is moderately loaded. When a node has a lot of tasks to execute, l will be low, thus, reducing its performance. Let M be the set of the adopted metrics, M = {mk , k = 1, 2, . . . , |M|} (e.g., M = {b, l }). For simplicity, we consider mk ∈ [0, 1]. Nodes monitor these metrics (e.g., b, l) to find the appropriate time to conclude P. For instance, when the node enjoys a high bandwidth and exhibits a low load, it can initiate P. In such cases, the execution of P will not disturb the node from the execution of the assigned tasks. In addition, we consider that nodes do not perform and adaptive modulation and coding and that the assigned tasks are not related to increased communication needs. Consider the discrete time T with t ∈ T (t is the variable representing the discrete time where nodes observe the environment and take decisions). Also, consider that a message indicating a new update has arrived in ni . A new UEj starts, UE j = [( j − 1 ) · T , j · T − 1] ∈ T. At each t ∈ UEj , ni checks each mk , k = 1, 2, . . . , |M| and calculates the ‘reward’ that will gain if it initiates P. The reward is calculated based on the observed performance metrics and it is proportionally or inverse proportionally affected by the type of each mk . For proportional metrics, the higher the value is, the higher the reward becomes. For inverse proportional metrics, the lower the value is, the higher the reward becomes. We define the function Ik that incorporates the information that a metric is proportional or not. The following equation stands true:



Ik =

mk 1 − mk

if mk is a proportional metric if mk is a non-proportional metric

(1)

Based on Ik , ni calculates the reward rt at t as follows: rt =  |M | I . rt gives to ni an indication about the current state of k=1 k the node and the network. Each ni has the aim to maximize rt and ‘safely’ conclude P. Based on rt , ni should the decide the appropriate time t∗ to initiate and conclude P. Hence, at t, ni should take one of the following decisions: 1 |M |

• D1. Stop the monitoring process, initiate and conclude P. • D2. Continue the monitoring process without deviating from the current task fulfilment. It should be noted that, if T expires, P is concluded immediately, however, ni gains no reward. 4. The time-optimized mechanism 4.1. Optimal stopping theory The OST [49] helps at determining the best time to take an action (decision) based on sequentially observed random variables. The final aim is to maximize the expected reward. The optimal stopping problem is defined by a sequence of random variables X1 , X2 , . . . whose joint distribution is known and a sequence of real-valued reward functions J0 , J (x1 ), J (x1 , x2 ), . . .. Let (, B, P) be the probability space, and Gt be the sub-σ -field of B generated by X1 , . . . , Xt . We have a sequence of σ -fields as G1 ⊂ G2 ⊂ . . . Gt ⊂ B. A stopping time is defined as a random variable ST ∈ 0, 1, . . . , ∞

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

5

Fig. 1. PDE of t∗ adopting the Uniform distribution.

such that the event {ST = t } is in Gt . The aim is to choose an optimal stopping time t∗ to maximize the expected reward E[Jt ∗ ]. When there is a known upper bound on the number of steps, it is a finite horizon problem and the optimal return can be solved by backward induction. If there is no bound on the number of steps at which one has to stop, this is an infinite horizon problem and the optimal return can be calculated through the optimality equation. 4.2. Model analysis Each node has an upper time limit denoted by T (finite horizon). At T, the node reaches the termination state where it has to conclude P. At t (recall that t is the variable representing the discrete time where each node observes the environment, calculates the reward and takes decisions), ni enjoys a ‘disturbance’ in the performance metrics as depicted by the random values corresponding to every mk . In this work, we focus on independent metrics and do not consider any adaptation process, especially in the underlying network to secure the aforementioned independence. The reward of ni at t is defined as yt = ft (yt−1 , rt ), t ∈ UE j , j = 1, 2, . . .. We define the function: yt = max{rt } as ni tries to find the maximum possible performance to conclude P. Based on yt , we consider the reward function at t as follows:



zt (yt , rt ) =

β t yt 0

if t ≤ T if t > T

(2)

It should be noted that no recall is permitted as ni cannot adopt any previous performance metrics realizations. The discount factor β ∈ (0, 1) affects ni ’s behaviour as follows. ni , probably, should delay the decision in anticipation of a better zt when β → 1. It should not delay the decision when the S requires an immediate conclusion of P (β → 0). If ni never stops, the reward is considered equal to zero, thus, we assume z0 = z∞ = 0. For identifying the optimal stopping time t∗ where the expected reward is maximized, we should focus on Z which is the random variable depicting ni ’s reward. Based on Z, we should find an optimal stopping rule, such that ni terminates the monitoring process to maximize the expected reward Zt , i.e., E[Zt ]. Once ni receives zt , it decides whether to continue the redmonitoring process or not, by examining the expectation of the future reward without recall. In other words, ni has to find an optimal stopping time (t∗ ) at which the supremum supt E[Zt ] is attained. Suppose that at some t, ni has received Zt = z and it is optimal to continue the monitoring process. Then, at the next stage t + 1, if Zt+1 is still z, because yt+1 ≤ z, it is optimal to continue due to the invariance of the problem in time [50]. Hence, based on the prin-

Fig. 2. PDE of t∗ adopting the Exponential distribution.

ciple of optimality, this problem can be solved as an optimal stopping problem with discounted future reward and without recall. This means that the reward defined by Eq. (2) can be considered as Zt = β t Yt = β t max(Rt ) and the problem assumes the same solution as the following problem: Find a t∗ such that the supt E[Zt ] is attained. Let rewards Z0 , Z1 , . . . , Z∞ where Zt = f (r1 , r2 , . . . , rt ). The sequence rt , Ft is defined by a probability space , an increasing sequence of sub σ -algebras {Ft }∞ 1 , the sequence of random variables Ri (ri ∈ Ft ) and E[ri ], ∀i. The following two assumptions should be true to have an optimal stopping time: (i) E[supt Zt ] <

6

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

Fig. 3. PDE of b∗ adopting the Uniform distribution.

∞; (ii) lim supt → ∞ Zt ≤ Z∞ . A stopping rule S∗ is the rule for which the following inequality stands true: E[ZS∗ ]|Ft ) > Zt a.s. on {S∗ > t}, ∀t. Concerning the existence of the optimal stopping rule, we rely on [50], where the following theorems have been proved: • Assuming condition (A), for any stopping rule S∗ , there is a regular stopping rule S such that E[ZS ] ≥ E[ZS∗ ]. • Under (A) and (B) conditions, there exists a stopping rule S∗ such that E[ZS∗ ] = M∗ where M∗ = supS E[ZS ]. • Under (A), if an optimal stopping rule exists, in particular, if (B) holds true, then S∗ is optimal. Theorem 1. For the model defined by Eq. (2), an optimal stopping time exists. Proof. have  Z ≤ β t max(r1 , r2 , . . . , rt ) ≤  We t j |r |. β Additionally, we have max β r1 , β 2 r2 , . . . , β t rt ≤ ∞ j j=1 ∞ β j E[supt Zt ] ≤ j=1 β E[|r j |] = 1−β E[|r j |] < ∞. Based on the above, condition (A) is satisfied.

Additionally, lim supt Zt ≤ limt β t

t j=1

|r j |

t

= limt β t

t

j=1 |r j |

t

.

From the law of large numbers, we take j=1 |r j | → E[|r |] and t β t = 0. Hence, limsupt Zt ≤ Z∞ = 0 and condition (B) is satisfied.  Based on the above, the optimal stopping rule and the optimal stopping time is given by the principle of optimality i.e., t ∗ = min {t ≥ 0 : Zt ≥ Z ∗ }. In the finite horizon case, there is an upper limit of stages equal to T. Till T, ni should conclude P. As the optimal stopping rule exists as well as the optimal stopping time t∗ , we can focus on the optimal stopping rule and solve our problem through backward induction. The solution in our problem is the solution of the following equation:



Zt (Yt ) =

max 0



 β t Yt , E(Zt+1 (Rt+1 ) )

if t ≤ T if t = T

(3)

Fig. 4. PDE of b∗ adopting the Exponential distribution.

Based on Eq. (3), the optimal strategy is: • Conclude P, if β t Yt > E(Zt+1 (Rt+1 ) • Continue the monitoring process, if β t Yt < E(Zt+1 (Rt+1 ) We take Jt =

Zt (Yt )

βt

and, thus, Jt (Yt ) = max(Yt , β E[Jt+1 (Rt+1 )] ). If

we take Jt (x ) = max(x, rt+1 ), we, finally, have that rt = β E[Jt (x )] = β E[max(x, rt+1 )]. Recall, that ri ∈ [0, 1], thus, the optimal stopping time is defined at the time where the current reward is greater than ωt . ωt is part of the recursive relation that defines the reward over which, ni should initiate and conclude P. The following equation stands true: ωt = β E(max(Yt , rt+1 ) ) and ωT = β E(R ).

4.3. Expected reward maximization For simplicity in our calculations, we focus on proportional parameters, i.e., b, l, that affect proportionally ni ’s reward. Our model depends on the selection of the probability distribution for variables Rt . In this section, we rely on two probability distributions and provide a solution for our model. We adopt: (a) the Uniform distribution and (b) the Exponential distribution. When applying the Uniform distribution, we assume that rt values are of equal probability in a specific interval. In other words, rt values have the

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

7

same probability to be observed by ni . On the other hand, by applying the Exponential distribution, we aim to handle various cases where ni assumes that performance metrics will be low or high as affected by the rate of the Exponential distribution. It should be noted that there is no reason to adopt a distribution ‘favourite’ to large values (rt → 1) as in these cases, the proposed mechanism is useless (nodes observed high bandwidth and low load). The reward rt is the convolution of two random variables each one defined in the interval [0,1], thus,

E(max(Yt , rt+1 ) ) =

 ωt+1 0

ωt+1 dF (r ) +



2

ωt+1

rdF (r ).

(4)

By solving Eq. (4), we can have the recursive relation that gives us the reward limit, for every t. Above this reward limit, ni stops observing rt and concludes P. Lemma 1. If mk∼ U(0, 1), with  k = 2, R ∼ U(0, 2) and the recursive β 3ω2 −3ω

+1

t+1 t+1 equation ωt = , ωT = β , defines values that indicate 6 the optimal stopping rule at t.

Proof. We consider two cases: (i) ωt+1 ≤ 1; (ii) ωt+1 > 1. In the former case (i.e., ωt+1 ≤ 1), we should find the solution to the fol ω 1 2 lowing equation: 0 t+1 ωt+1 rdr + ω r 2 dr + 1 r (2 − r )dr. By solvt+1

ω3

ing the integrals, we get the equation t+1 6 + 1. In the latter case (i.e., ωt+1 > 1), we should find the solution to the following 1 ω 2 equation: 0 ωt+1 rdr + 1 t+1 r (2 − r )dr + ω r (2 − r )dr. By solvt+1

3 ωt+1

2 ing the integrals, we get the equation − 6 + ωt+1 − ωt+1 + 43 . The two cases are of equal probability and, thus, we get that  

ωt =

2 −3ω β 3ωt+1 t+1 +1

6

.



Lemma 2. If mk ∼ Exp(λk ), with k = 2, R ∼ Hypo(λ1 , λ2 , . . . ) [51] and the recursive equation ωt = λ −β λ (a + b)



2

1

a = ωt+1 λ1 e−λ2 ωt+1 − λ2 e−λ1 ωt+1 + λ2 − λ1 , λ1 λ2 ( f (ωt+1 , λ1 ) − f (ωt+1 , λ2 ) + g(ωt+1 , λ2 ) − g(ωt+1 , λ1 ) ), with

ωT =

b=

(λωt+1 +1 )e−λωt+1 β ( λ1 + λ 2 ) ) e −λ and g(λ ) = (λ+1 , λ1 λ2 , f (ωt+1 , λ ) = λ2 λ2

defines values that indicate the optimal stopping rule at t.

Proof. The Probability Density Function (PDF) hR of the random variable R i.e., a sum of exponentials with different rates is given as follows [52]:

hR (t ) =

|M|

i=1

λ1 . . . λ|M|  |M|

eλi t I0,∞ (t )

(5)

j=1, j=i

∀t ∈ T. By applying hR in Eq. (4) and through calculations we conclude the recursive equation as the Lemma indicates.



Fig. 5. PDE of l∗ adopting the Uniform distribution.

distribution is the basis for the Exponential family (e.g., Normal, Gamma, Chi-squared) and it exhibits an important property i.e., it is memoryless. Future events are independent of past events. This is very important in our setting as we want to simulate scenarios where performance parameters realizations may not have any relation to historical values. Hence, based on these two distributions we could cover multiple scenarios that could depict real settings. When the Uniform distribution is adopted, we aim to handle scenarios where ni monitors the system in an ‘agnostic’ manner. The term ‘agnostic’ means that ni considers that b and l have an equal opportunity of occurring. When we adopt the Exponential distribution, we focus on high or low values of b and l depending on the λ parameter. For instance, when λ = 2.0, ni monitors low values for b and l. The higher the λ is, the smaller the values become. In this setting, we aim to evaluate different scenarios where the dynamic environment affects b, l, thus, the decision of ni . When λ → 1.0, we assume that data related to network and node’s performance exhibit small ‘fluctuations’ and ni could easily conclude high performance values for b, l. When λ is very high, we assume that the monitored performance exhibits ‘heavy fluctuations’ and ni cannot easily monitor and conclude high performance values. b and l parameters get their values by adopting the PDF of the exponential distribution, i.e., f (x ) = λe−λx . The interpretation of each value is based on Eq. (1) depending on the ‘type’ of the parameter (proportional or non proportional). Each performance metric realization is adopted for the calculation of Ik and, afterwards, the calculation of the reward that nodes enjoy.

5. Performance evaluation

5.2. Evaluation metrics & simulation setup

5.1. Methodology and experimental setup

We report on the performance of the proposed time-optimized scheme concerning two axes: (i) the time required for deciding the initiation of P and (ii) the quality of the decision in terms of the network and node’s performance. Our model aims to identify the highest possible value for the observed metrics that secures the best performance. We report on the optimal stopping time t∗ and the value of b and l when the stopping decision is realized, i.e., b∗ , l∗ . The optimal performance is realized when t∗ → 0 and b∗ , l∗ are the maximum possible for the specific update era UEj . Without loss of generality, in our experiments, we consider b, l ∈ [0, 1], thus, the optimal performance is achieved when b∗ → 1 and l∗ → 1. In Table 1, we present the adopted evaluation metrics accompanied with a short description. We define the τ metric to depict the time required to conclude a stopping decision. The following equation holds true:

We elaborate on the performance of the proposed model. Through a large number of simulations, we evaluate the finite horizonTime-optimized Update Management Process (TUMP). The simulator is written in Java and executed in a PC with Intel Pentium G2130 3.2Ghz CPU, 4GB RAM running Windows 7. Various simulation scenarios are adopted to evaluate the performance of the TUMP when we adopt the Uniform or the Exponential distribution for depicting the values of b and l. The reasons for adopting these two distributions are: (a) the Uniform distribution is the simplest, however, any other distribution can be modeled by distorting it based on the technique of the inverse transform sampling [34]. Hence, we can easily depict multiple simulation scenarios that could closely cover real world settings; (b) the Exponential

8

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

Fig. 7. Results for γ , δ , τ adopting the Uniform distribution.

τ=

t∗ . T

(6)

τ depicts the amount of time required to conclude P out of the deadline T. As τ → 1, ni reaches the available deadline to apply P. Such a behaviour is not efficient as we want the nodes immediately concluding P. When τ → 0, ni identifies a high value for b and l at the first rounds of the monitoring process. The quality of the result (i.e., the maximum possible b∗ and l∗ for the era UEj ) is evaluated by the γ and δ metrics. The following equations hold true:

γ=

δ=

Fig. 6. PDE of l∗ adopting the Exponential distribution.

Table 1 Performance evaluation metrics. Metric

Short description

τ γ δ

the time required to conclude the stopping decision the average bandwidth measured at the stopping decision the average ‘availability’ of each node at the stopping decision the difference in the performance between the proposed models and other relevant schemes in the literature

D

E

i=1

b∗

E E

i=1

E

l∗

,

,

(7)

(8)

where E is the number of the experiments. γ depicts the average bandwidth for every experiment while δ depicts the average “availability” of each node. The higher the γ and the δ are, the better performance our model exhibits. The optimal performance is achieved when γ → 1.0, δ → 1.0. To the best of our knowledge, the proposed scheme is one of the first attempts to model the behavior of a node in the direction to find when to initiate the update process. The remaining models found in the literature deal with how to apply the update process. We compare the proposed model (i.e., TUMP) with a deterministic model DM. In the DM, the behaviour of nodes is exactly predictable (i.e., conclude P immediately) when the update messages are received [53]. The DM concludes P immediately when an update message is sent by the server only if b and l are over a pre-defined threshold (to secure the smooth conclusion of P). The DM is the theoretical limit for our model concerning the time spent to derive a decision for the conclusion of P in combination with securing a relatively high bandwidth and availability. In addition, the DM incorporates the risk of not exceeding the pre-defined thresholds, especially, in noisy and low performance networks. In such cases, P will be concluded at T. We also compare our model with the Moving Average Estimator (MAE) [54] and the Exponentially Weighted Moving Average (EWMA) estimator [54]. MAE is a simple estimator that is widely used in measuring the link quality of wireless networks. The algorithm performs on top of a window of historical values and derives the average value. When the average is over a pre-defined threshold MAE initiates and concludes P. EWMA estimator is memory efficient, requiring constant storage of

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

9

Fig. 9. TUMP - DM comparison adopting the Uniform distribution.

lowing equation holds true:

D=

PT UMP − PREF · 100%, PREF

(9)

where PTUMP is the performance of the TUMP and PREF is the performance of the reference model (i.e., REF ∈ {DM, MAE, EWMA}). D is realized for each performance metric i.e., γ , δ , τ and shows if the TUMP performs better than the reference models DM, MAE, EWMA. For metrics γ and δ , when D is positive means that the TUMP performs better than the reference model while the opposite stands when D is negative. The higher the D is, the better the performance of the TUMP becomes. For τ , when D is negative means that the TUMP performs better than the reference model while the opposite stands when D is positive. The lower the D is, the less the time required by TUMP to conclude P compared to the remaining models. We evaluate our model for diverse realizations of β and λ1 , λ2 , i.e., β ∈ {0.2, 0.5, 0.95} and λ1 , λ2 ∈ {1, 5}. The greater the β is, the greater the loss / cost becomes. The cost is realized as the cost of observing a value at every round of the era UEj and the loss that ni enjoys when selecting values with low expected ranks. In addition, the higher the λ1 and λ2 are, the smaller the values of b and l become. An increased λ simulates the scenario where the bandwidth and the availability of each node are low. In such cases, it is difficult to enjoy a high quality performance to conclude P. We also consider different scenarios for b and l by incorporating a set of λ1 , λ2 combinations i.e., λ1 = λ2 = 1, λ1 = λ2 = 5, λ1 = 5, λ2 = 1, λ1 = 1, λ2 = 5. Through this approach, we try to reveal the performance of the TUMP for various combinations of b and l. At each t ≤ T, we randomly generate b and l and apply our model to record t∗ , b∗ and l∗ . 5.3. Performance assessment

Fig. 8. Results for γ , δ , τ adopting the Exponential distribution.

the old estimate for any kind of history tuning. EWMA adopts a linear combination of infinite history, weighted exponentially. Again, when the EWMA’s result is over a pre-defined threshold, P is realized. We also define the difference D for each performance metric discussed above to depict the difference in the performance between the proposed model TUMP and DM, MAE, EWMA. The fol-

We report on the Probability Density Estimation (PDE) of t∗ , b∗ and l∗ . In Fig. 1, we plot the PDE(t∗ ) for β ∈ {0.2, 0.5, 0.95}. The lower the β is, the lower the t∗ becomes. This is natural because the node enjoys an increased cost for the monitoring process. Hence, the node concludes P immediately. We also observe that a high β leads to high t∗ as the monitoring cost is low (when β → 1), thus, the node can wait for more and conclude P for the highest possible values of b and l. The aforementioned results concern the scenario where the Uniform distribution expresses the realization of both b and l. Let us now have b and l characterized by the Exponential distribution. Fig. 2 presents our results. It should be noted that any difference in t∗ concerns the scenario where

10

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

Fig. 11. TUMP - MAE comparison adopting the Uniform distribution.

Fig. 10. TUMP - DM comparison adopting the Exponential distribution.

β = 0.2. In the remaining cases (i.e., β ∈ {0.5, 0.95}, we do not observe any important changes in t∗ no matter the values of λ1 and λ2 . The reason is that our model is mainly affected by the monitoring cost that presses nodes to conclude the update process immediately and not by the observed values. In Figs. 3 and 4, we depict our results concerning b∗ . Recall that the higher the b∗ is, the higher the performance of the network

becomes. A high b∗ will facilitate the node to smoothly download and apply the desired updates. Fig. 3 shows our results when the Uniform distribution is the case. We observe that the higher the β is, the higher the b∗ becomes. A high β makes a node to pursue high values for b and l even if the node consumes more time to conclude P (see Fig. 1). In the case of the Exponential distribution (Fig. 4), we observe that that λ1 and λ2 do not significantly affect the performance of the model. In addition, it seems that TUMP exhibits the same performance not matter the β . TUMP enjoys similar bandwidth realizations at the stopping decision as such a decision is taken in combination with the l realizations. This means that the focus of the proposed model is not derived exclusively on the bandwidth but also in the availability of the node to perform tasks. The decision making tries to pay equal attention on both parameters, i.e., b and l before deciding to conclude P. In Figs. 5 and 6, we depict our results concerning l∗ . We observe similar results as in the experimental outcomes for b∗ . Fig. 5 shows our results when the Uniform distribution is adopted while Fig. 6 depicts our results for the Exponential distribution. We observe that the higher the β is the higher the l∗ becomes. In the case of the Exponential distribution (Fig. 6), we observe that that λ1 and λ2 do not significantly affect the performance of the model as already discussed for the outcomes related to b∗ . In Fig. 7, we plot γ , δ and τ results for the Uniform distribution. As β → 1, γ , δ and τ increase as well. This means that the node enjoys high b∗ and l∗ , however, for increased τ (time to take the final decision). The limited cost paid for monitoring (when β → 1) makes the node to wait till increased b and l are observed just before deciding to stop the monitoring process and conclude P. The TUMP results an increment of 48%, 48% and 230% for γ , δ and τ , respectively, when β is increased from 0.2 to 0.95 (Fig. 8). The highest increment is observed for τ . When a node adopts β = 0.95, it tries to pursue high values for the performance parameters, thus, it spends more time in the monitoring process expecting high values at the upcoming rounds. When the Exponential distribution is under consideration, we observe that β is the parameter that significantly affects our scheme. γ and δ take low values compared to the results related to the Uniform distribution. Our model exhibits better performance when the monitored data are characterized by many fluctuations due to the randomness of the observations. Recall that the Exponential distribution exhibits an attitude to low or high values depending on the λ we choose. Hence, a high number of fluctuations is eliminated. We report on the comparison of the TUMP with the reference models based on the metric D as defined in Eq (9) with

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

11

Fig. 13. TUMP - EWMA comparison adopting the Uniform distribution.

Fig. 12. TUMP - MAE comparison adopting the Exponential distribution.

REF ∈ {DM, MAE, EWMA}. In Figs. 9 and 10, we plot our results for the T UMP − DM comparison (Fig. 9 results are derived by adopting the Uniform distribution and Fig. 10 reports on the derived results adopting the Exponential distribution). In Fig. 9, we observe that TUMP outperforms the DM only when β = 0.95 (the cost paid for the monitoring process is low). This result holds true for the γ and δ metrics. In addition, the TUMP outperforms the DM concerning the τ metric no matter the β value. The DM secures that b and l will exceed the pre-defined threshold (i.e., 0.80), however, requiring more observations before the final decision is concluded. In the Exponential distribution case, we observe different results according to the β value. The TUMP outperforms the DM concerning the τ metric no matter the β . When β < 0.95, DM performs better than the TUMP for the remaining metrics. The DM cannot easily conclude a reward for the node, on top of the observed b and l, above the pre-defined threshold when β = 0.95 and the data are characterized by the Exponential distribution. In contrast, the proposed TUMP has the ability to decide in relatively short time the final decision with b∗ and l∗ better than the DM. The comparison between the TUMP and the MAE (Figs. 11 and 12) lead to similar results with the comparison between TUMP DM. The TUMP outperforms the MAE in specific cases (β → 0.95) concerning γ and δ . In addition, the MAE requires more time than the TUMP to reach the final decision no matter the cost of observations. In the Exponential distribution case, the TUMP results better than the MAE when β = 0.95, i.e., the averaging mechanism defined by the MAE cannot conclude consecutive values above the pre-defined threshold. If we choose to reduce the threshold, naturally, the quality of the decision, as realized by the b∗ and l∗ , will be minimized. Finally, in Figs. 13 and 14, we plot our results concerning the comparison between TUMP - EWMA. When the Uniform distribution is adopted, the TUMP performs better than the EWMA for β = 0.95 (γ and δ metrics). In addition, the time required by the TUMP is lower than the EWMA no matter the β . In contrast to the aforementioned comparisons (i.e., TUMP - DM, TUMP - MAE), the TUMP results better than the EWMA also when β = 0.5 concerning the δ metric. When we adopt the Exponential distribution (Fig. 14), the TUMP results better than the EWMA when β ≥ 0.5. The difference in the performance becomes high when β = 0.95. We also observe that when b is characterized by high values (λ1 → 1) and l by low values (λ2 = 5) the difference becomes more intense for γ and δ compared to the remaining experimental scenarios.

12

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14

finding the appropriate algorithms and protocols to distribute the updates while being aware of nodes specific characteristics. In our scheme, each node, independently, decides when it will conclude the update according to the result of a monitoring process. The monitoring process aims to provide a view on the performance of the network and the node itself. We envision that each node monitors specific metrics and tries to identify when the performance is of high quality. When the performance is of high quality, there is room for applying the updates without disturbing nodes from the assigned tasks. The proposed time-optimized, performance aware mechanism makes the nodes to be aware of their current internal state while being adapted to the network performance. In contrast to centralized systems, the network is not flooded by update messages and nodes’ performance remains at high levels. We adopt a finite horizon OST model applied on top of multiple performance metrics to conclude a mechanism where the identification of the highest possible metrics realizations (as depicted by real numbers) is the primary goal. The model results the time when the update process should be concluded. Finally, the proposed mechanism is fully applied in the performance of the network securing the uninterrupted application of the updates. Future extensions of our work involve the implication of an infinite horizon model where the pressure of time for applying the updates is absent. Usually, such a setting involves updates related to peripheral issues like software extensions. With this approach, we will offer a complete model for concluding updates either in short- (finite horizon) or in long-term (infinite horizon).

References

Fig. 14. TUMP - EWMA comparison adopting the Exponential distribution.

6. Conclusions & future work The new era of the IoT demands intelligent applications on top of the autonomous nature of independent nodes. In this paper, we elaborate on a distributed, time-optimized, performance aware model that aims to help the autonomous nodes to initiate and conclude update processes when working in the network. The proposed scheme alleviates the central servers from the burden of

[1] L. Atzori, A. Iera, G. Morabito, The Internet of Things: A Survey, in: Computer Networks, Elsevier, 2010, pp. 2787–2805. [2] J.W. Hui, D. Culler, The dynamic behavior of a data dissemination protocol for network programming at scale, in: Proceedings of the International Conference on Embedded networked sensor systems, SenSys, Baltimore, MD, USA, 2004, pp. 81–94. [3] I.H. Hou, Y.E. Tsai, T.F. Abdelzaher, I. Gupta, Adapcode: adaptive network coding for code updates in wireless sensor networks, in: Proceedings of the IEEE 27th Conference on Computer Communications, Phoenix, AZ, USA, 2008, pp. 2189–2197. [4] C. Intangonwiwat, R. Govindan, D. Estrin, Directed diffusion: a scalable and robust communication paradigm for sensor networks, in: Proceedings of the 6th Annual Conference on Mobile Computing and Networking, Boston, MA, USA, 20 0 0, pp. 56–67. 611 August. [5] S. Kulkarni, L. Wang, Mnp: multihop network reprogramming service for sensor networks, in: Proceedings of the 25th IEEE International Conference on Distributed Computing Systems, Columbus, OH, USA, 2005, pp. 7–16. [6] S.S. Kulkarni, M. Arumugam, Infuse: A TDMA Based Data Dissemination Protocol for Sensor Networks, Michigan State University: East Lansing, MI, USA, 2004 Technical report msu-cse-04-46 for the department of computer science and engineering. [7] H. Leligou, C. Massouros, E. Tsampasis, T. Zahariadis, D. Bargiotas, K. Papadopoulos, S. Voliotis, Reprogramming wireless sensor nodes, Int. J. Comput. Trends Technol., Seventh Sense Res. Grp. (2011) 1–8. [8] P. Levis, N. Patel, D. Culler, S. Shenker, Trickle: a self-regulating algorithm for code propagation and maintenance in wireless sensor networks, in: Proceedings of the Symposium on Networked Systems Design and Implementation, vol. 1, San Francisco, California, 2004, p. 2. [9] C. Miller, C. Poellabauer, Reliable and efficient reprogramming in sensor networks, ACM Trans. Sens. Netw. 7 (2010) 1–32. [10] V. Naik, A. Arora, P. Sinha, H. Zhang, Sprinkler: a reliable and energy efficient data dissemination service for wireless embedded devices, in: Proceedings of the 26th IEEE International Real-Time Systems Symposium, Miami, FL, USA, 2005, pp. 286–296. 58 December. [11] R. Panta, I. Khalil, S. Bagchi, Stream: low overhead wireless reprogramming for sensor networks, in: Proceedings of the International Conference on Computer Communications, INFOCOM, Barcelona, Spain, 2007, pp. 928–936. [12] H.U. Park, J. Jeong, P. mah, Non-invasive rapid and efficient firmware update for wireless sensor networks, in: Proceedings of the UBICOMP, Seattle, WA, USA, 2014, pp. 147–150. [13] F. Stann, J. Heidemann, RMST: reliable data transport in sensor networks, in: Proceedings of the 1st IEEE International Workshop on Sensor Network Applications and Protocols, Anchorage, AK, USA, 2003, pp. 102–112. 11 May. [14] T. Stathopoulos, J. Heidemann, D. Estrin, A remote code update mechanism for wireless sensor networks, Technical Report, Center for Embedded Networked Sensing, 2003.

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14 [15] C.Y. Wan, A.T. Campbell, L. Krishnamurthy, PSFQ: a reliable transport protocol for wireless sensor networks, in: Proceedings of the 1st ACM international workshop on Wireless sensor networks and applications, Atlanta, Georgia, USA, 2002, pp. 1–11. 28 September. [16] V. Raychoudhury, J. Cao, M. Kumar, D. Zhang, Middleware for Pervasive Computing: A Survey, in: Pervasive and Mobile Computing, 9, Elsevier, 2013, pp. 177–200. [17] P.P. Ray, A Survey on Internet of Things Arcitectures, Journal of King Saud University - Computer and Information Sciences, Elsevier, 2016. 10.1016/j.jksuci. 2016.10.003. [18] S. Brown, C. Sreenan, Software updating in wireless sensor networks: a survey and lacunae, J. Sensor Actuators, Els. 2 (2013) 717–760. [19] H.Y. Shwe, F. Adachi, Power efficient adaptive network coding in wireless sensor networks, in: Proceedings of the IEEE International Conference on Communications (ICC), Kyoto, Japan, 2011. 10.1109/icc.2011.5963363. [20] P. Levis, S. Madden, J. Polastre, R. Szewczyk, A. Woo, D. Gay, J. Hill, M. Welsh, E. Brewer, D. Culler, TinyOS: an operating system for sensor networks, in: W. Weber, J. Rabaey, E. Aarts (Eds.), Ambient Intelligence, Springer, Berlin, Heidelberg, 2005, pp. 115–148. [21] N. Tsiftes, A. Dunkels, T. Voigt, Efficient sensor network reprogramming through compression of executable modules, in: Proceedings of the 5th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, San Francisco, California, USA, 2008. 10.1109/ SAHCN.2008.51. [22] M. Stolikj, P. Cuijpers, J. Lukkien, Efficient reprogramming of wireless sensor networks using incremental updates and data compression, in: Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops, San Diego, CA, USA, 2013, pp. 584–589. [23] A. Dunkels, B. Grnvall, T. Voigt, Contiki - a lightweight and flexible operating system for tiny networked sensors, in: Proceedings of the 29th Annual IEEE International Conference on Local Computer Networks, Tampa, FL, USA, USA, 2004, pp. 455–462. [24] Q. Cao, T. Abdelzaher, J. Stankovic, T. He, The liteos operating system: towards unix-like abstractions for wireless sensor networks, in: Proceedings of the 7th international conference on Information processing in sensor networks, IPSN ’08, St. Louis, MO, USA, USA, 2008, pp. 233–244. [25] H. Cha, S. Choi, I. Jung, H. Kim, H. Shin, J. Yoo, C. Yoon, RETOS: resilient, expandable, and threaded operating system for wireless sensor networks, in: Proceedings of the 6th International Conference on Information processing in sensor networks, IPSN ’07, Cambridge, MA, USA, 2007, pp. 148–157. [26] T. Istomin, A.L. Murphy, G.P. Picco, U. Raza, Data prediction + synchronous transmissions = ultra-low power wireless sensor networks, in: Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems, Stanford, CA, USA, 2016, pp. 83–95. [27] S. Saginbekov, A. Jhumka, Efficient code dissemination in wireless sensor networks, Future Gener. Comput. Syst., Els. 39 (2014) 111–119. [28] E. Yoneki, J. Bacon, Unified semantics for event correlation over time and space in hybrid network environments, in: Proceedings of the IFIP International Conference on Cooperative Information Systems, Cyprus, 2005, pp. 366–384. [29] E. Souto, G. Guimaraes, G. Vasconcelos, M. Vieira, N. Rosa, C. Ferraz, A message-oriented middleware for sensor networks, in: In Proceedings of the 2nd International Workshop on Middleware for Pervasive and Ad-Hoc Computing, Toronto, Canada, 2004, pp. 127–134. [30] W. Heinzelman, A. Murphy, H. Carvalho, M. Perillo, Middleware to support sensor network applications, IEEE Netw. 1 (18) (2004) 6–114. [31] N. Xue, L. Liang, J. Zhang, X. Huang, POSTER: a framework for iot reprogramming, in: R. Deng, J. Weng, K. Ren, V. Yegneswaran (Eds.), Security and Privacy in Communication Networks, SecureComm 2016, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 198, Springer, Cham, 2017. [32] K. Lin, P. Levis, Data discovery and dissemination with dip, in: Proceedings of the International Conference on Information Processing in Sensor Networks (IPSN 2008), Washington, DC, USA, 2008, pp. 433–444. [33] T. Dang, N. Bulusu, W. Feng, S. Park, DHV: a code consistency maintenance protocol for wireless sensor networks, in: Proceedings of the 6th European Conference on Wireless Sensor Networks (EWSN 2009), Cork, Ireland, 2009, pp. 327–342. [34] L. Devroye, Non-Uniform Random Variate Generation, Springer-Verlag, New York, 1986.

13

[35] W. Li, Y. Zhang, B. Childers, MCP: an energy-efficient code distribution protocol for multi-application WSNs, in: Proceedings of the 5th IEEE International Conference on Distributed Computing in Sensor Systems, Marina del Rey, CA, USA, 2009, pp. 259–272. [36] K. Maier, A. Hessler, O. Ugus, J. Keller, D. Westhoff, Multi-hop over-the-air reprogramming of wireless sensor networks using fuzzy control and fountain codes, Self-Organising Wireless Sensor and Communication Networks, Hamburg, Germany, 2009. [37] Y. Yu, L.J. Rittle, V. Bhandari, J.B. Lebrun, Supporting concurrent applications in wireless sensor networks, in: Proceedings of the 4th International Conference on Embedded Networked Sensor systems, SenSys, Boulder, Colorado, USA, 2006, pp. 139–152. [38] C. Dong, F. Yu, An efficient network reprogramming protocol for wireless sensor networks, Comput. Commun., Elsevier 55 (2015) 41–50. [39] R. Crepaldi, A. Harris, M. Rossi, G. Zanca, M. Zorzi, Fountain reprogramming protocol (frp): a reliable data dissemination scheme for wireless sensor networks using fountain codes, in: Proceedings of the 5th International Conference on Embedded Networked Sensor Systems, ACM, Sydney, Australia, 2007, pp. 389–390. [40] M. Rossi, G. Zanca, L. Stabellini, R. Crepaldi, A. Harris, M. Zorzi, Synapse: a network reprogramming protocol for wireless sensor networks using fountain codes, in: Proceedings of the 5th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, IEEE, San Francisco, California, USA, 2008, pp. 188–196. [41] S.E. Ha, S. Choi, B. Song, H.S. Lee, A new data propagation scheme for WSN network reprogramming in a noise-full environments, in: Proceedings of the 2011 International Conference on Information Networking (ICOIN), IEEE Kuala Lumpur, Malaysia, 2011, pp. 547–552. [42] A. Hagedorn, D. Starobinski, A. Trachtenberg, Rateless deluge: over-the-air programming of wireless sensor networks using random linear codes, in: Proceedings of the International Conference on Information Processing in Sensor Networks, St. Louis, Missouri, USA, IEEE, 2008, pp. 457–466. [43] T. Ho, M. Mdard, R. Koetter, D.R. Karger, M. Effros, J. Shi, B. Leong, A random linear network coding approach to multicast, IEEE Trans. Inf. Theory 52 (2006) 4413–4430. [44] K. Alhamed, Framework for Open-Source Software Update Mechanism: Resistance to Stacking the Deck Attack, Phd thesis, Florida Institute of Technology, 2015. [45] W. Itani, A. Kayssi, A. Chehab, PETRA: a secure and energy-efficient software update protocol for severely-constrained network devices, in: Proceedings of the 5th ACM symposium on QoS and security for wireless and mobile networks, Tenerife, Canary Islands, Spain, 2009, pp. 37–43. [46] F. Gozzi, Optimal control problems: the dynamic programming approach. Short Course, Politechnico di Torino, retrieved at Sept. 20th, 2017 from http://calvino. polito.it/∼probstat/Gozzi_Dispense.pdf. [47] A. Hava, Y. Ghamri-Doudane, J. Murphy, A study on monitoring overhead impact on wireless mesh networks, in: Proceedings of the 8th International Wireless Communications and Mobile Computing Conference, IWCMC12, Limassol, Cyprus, 2012, pp. 487–492. [48] A. Hanemann, A. Liakopoulos, M. Molina, D.M. Swany, A study on network performance metrics and their composition, Campus-Wide Inf. Syst., Emelard Insight 23 (4) (2006) 268–282. [49] G. Peskir, A. Shiryaev, Optimal Stopping and Free Boundary Problems, ETH Zuerich, Birkhauser, 2006. [50] T.S. Ferguson, Optimal stopping and applications, Mathematics Department, UCLA, 2016. Available online http://www.math.ucla.edu/tom/Stopping/ Contents.html, accessed March. [51] A. Sen, N. Balakrishnan, Convolution of geometrics and a reliability problem, Stat. Probab. Lett., Elsevier 43 (1999) 421–426. [52] M. Akkouchi, On the convolution of exponential distributions, J. Chungcheong Math. Soc., Korea Sci. 21 (4) (2008) 501–510. [53] A. Gelman, I. Leenen, I. Van Mechelen, P. De Boeck, J. Poblome, Bridges between deterministic and probabilistic models for binary data, Stat. Methodol., Elsevier 7 (2010) 187–209. [54] A. Woo, D. Culler, Evaluation of efficient link reliability estimators for low-power wireless networks, EECS Department, University of California, Berkeley, 2003 Tech. rep. ucb/csd-03-1270.

14

K. Kolomvatsos / Ad Hoc Networks 69 (2018) 1–14 Kostas Kolomvatsos received his B.Sc. in Informatics from the Dept. of Informatics at the Athens University of Economics and Business in 1995, his M.Sc. and his Ph.D. in Computer Science from the Dept. of Informatics and Telecommunications at the University of Athens (UoA) in 2005 and in the beginning of 2013, respectively. Currently, he serves as a Senior Researcher in UoA and as an Adjunct Lecturer at the University of Thessaly, Dept. of Computer Science. He has participated in several European and National research projects. His research interests are in the definition of Intelligent Systems adopting Machine Learning, Computational Intelligence and Soft Computing for Pervasive Computing, Distributed Systems and Large Scale Data Streams. He is the author of over 45 publications in these areas.