“Slow is Fast” for wireless sensor networks in the presence of message losses

J. Parallel Distrib. Comput. 77 (2015) 41–57

Contents lists available at ScienceDirect

J. Parallel Distrib. Comput. journal homepage: www.elsevier.com/locate/jpdc

‘‘Slow is Fast’’ for wireless sensor networks in the presence of message losses Reza Hajisheykhi a,∗ , Ling Zhu a , Mahesh Arumugam b , Murat Demirbas c , Sandeep Kulkarni a a

Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, USA

b

Citrix Systems, Inc., Santa Clara, CA, 95054, USA

c

Computer Science and Engineering Department, SUNY at Buffalo, Buffalo, NY, 14260, USA

highlights • • • • •

None of the existing computational models consider message loss/collision in the distributed systems. WAC model is a model that considers message loss in distributed systems. However, it reduces the performance. Our work is a variation of the shared memory model, namely SF shared memory model. It can improve the performance in the presence of message loss. We present an analytical proof (and evaluations for three protocols) for our SF model.

article

info

Article history: Received 12 December 2012 Received in revised form 16 October 2014 Accepted 19 November 2014 Available online 10 December 2014 Keywords: Wireless sensor networks Shared memory model Self-stabilization

abstract We present a new shared memory model, SF shared memory model. In this model, the actions of each node are partitioned into slow actions and fast actions. By contrast, the traditional shared memory model only includes fast actions. Intuitively, slow actions can utilize slightly stale state information to execute successfully. However, fast actions require that the state information they use is most recent. We show that the use of slow actions can substantially benefit in improving performance of programs from the shared memory model to WAC model that has been designed for sensor networks. To illustrate this, we use three protocols concerning problems that need to be solved in sensor networks. We show that under various message loss probabilities, densities, etc., slow actions can improve the performance substantially, since slow actions reduce the performance penalty of fast actions under heavy message loss environments. Moreover, the effectiveness of the slow action increases when there is a higher probability of message loss. © 2014 Elsevier Inc. All rights reserved.

1. Introduction Several computation models have been proposed for distributed computing, including shared memory model, read/write model, and message passing model. These models differ in the level of abstraction they provide. Low level models such as the message passing model permit one to write programs that are closer to the actual system implementation and, hence, the programs can potentially be implemented more efficiently. However, since such programs need to analyze low level communication issues such

∗

Corresponding author. E-mail address: [email protected] (R. Hajisheykhi).

http://dx.doi.org/10.1016/j.jpdc.2014.11.004 0743-7315/© 2014 Elsevier Inc. All rights reserved.

as channel contention, message delays, etc., they are difficult to design and verify. Using a high level abstraction enables the designers to ignore low-level details such as process communication and facilitates the design and verification of the protocols. For example, shared memory model, which allows a node to simultaneously read all its neighbors and update its own state, has been used extensively in the distributed systems literature. The drawback of using a high level abstraction model is that the system implementation requires more effort. Moreover, neither of the previously mentioned low level nor high level models consider message loss and message collision in the distributed systems. Transformations from shared memory model to read/write model or message passing model have been considered in [12,28,30]. Wireless sensor networks (WSNs) warrant a new computation model due to their wireless broadcast communication mode, not

42

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

captured in any of the above-mentioned models. Write-all-withcollision (WAC) model has been proposed in [19] to capture the important features of wireless broadcast communication for WSNs. In this model, in one step, a node can write its own state and communicate it to its neighbors. Due to the nature of shared medium, if one node is being updated by two (or more) of its neighbors simultaneously, the update fails (message collisions leads to message loss). While WAC model enables us to analyze the energy efficiency and message cost of protocols in WSNs more easily, it is not straightforward to design and verify protocols in WAC model compared to a higher level model such as the shared memory model. Transformations from shared memory model to WAC model have been studied in [19,22]. Although the shared memory model and the WAC model are similar in principle (in that the former allows a node to read all its neighbors whereas the latter allows the node to write to all its neighbors), direct transformation becomes inefficient when message losses are considered. Specifically, in [19], a CSMA based transformation, Cached Sensornet Transform (CST), from shared memory model to WAC model has been presented. In CST, a single message loss may violate the correctness of the resultant concrete program in the WAC model. The proof in [19] shows that if the abstract program was designed to be self-stabilizing and no other message loss occurs for a sufficiently long period, the concrete program will stabilize and start making progress. Thus, given the message loss rates at WSNs this transformation incurs heavy correctness and performance loss at the WAC level. In [22], a transformation from read/write model to WAC model has been presented, and as we show in Section 2.1, it also applies for transformation from shared memory model to WAC model. This transformation employs a TDMA schedule to reduce message losses in the WAC model. However, due to interference, fading, or sleeping nodes, message losses are still likely in the real deployment. Message losses do not violate safety in this transformation, but they reduce the performance because the loss of a broadcast from a node prevents the evaluation of the actions at other nodes that depend on that information. To mitigate the performance problems, we propose a variation of the shared memory model, namely SF shared memory model. Similar to the shared memory model, in the SF shared memory model, in each step, a node is allowed to read the state of its neighbors and write its own state. However, in the SF model, actions of each node are partitioned into ‘slow’ actions and ‘fast’ actions. If a node j determines that a fast action is enabled then j must execute the fast action immediately before j’s neighbors change their state. If j fails to execute the fast action immediately, then j must verify whether that action is still enabled the next time j evaluates its guards. On the other hand, if j determines that a slow action is enabled, then j can execute the slow action at any point later as long as j does not execute any other action in between. Thus, a slow action is especially useful if j can determine that the guard of an action was true sometime in the past. By contrast, this knowledge is not useful in execution of fast actions. We point out that the use of SF model can substantially improve the performance of the transformed programs in the presence of message loss. Specifically, the contributions of the paper are as follows. Contributions of the paper.

• We show that, in the presence of message loss, the use of SF shared memory model improves the performance of the transformed program substantially. • We present guidelines for the protocol designer to identify slow and fast actions.

• We consider three commonly used protocols to identify slow and fast actions. These protocols include tree construction, distributed reset and asynchronous unison. Of these, the tree construction protocol typically forms a backbone of routing and data propagation. The distributed reset mechanism is useful in reorienting the given network to perform new tasks as well as to recover the network from loss of coordination. And, asynchronous unison is an abstract version of clock synchronization where the clocks in a given neighborhood are kept close to each other. Thus, asynchronous unison can be used to correlate observations at different nodes. • We present an analytical comparison of slow and fast actions in the SF model. The analytical reasoning demonstrates that slow actions can substantially improve protocol performance in the presence of message loss. • We evaluate the benefit of slow actions in the three protocols described above. These simulation results with the help of ProSe [7] and TOSSIM [25] validate the analytical results. • We introduce the notion of pseudo-slow actions. Intuitively, pseudo-slow actions correspond to slow-motion execution of fast actions. In other words, if a fast action is believed to be highly important for performance of the given protocol, this approach has the potential to improve the effectiveness of executing that fast action. Organization of the paper. The rest of the paper is organized as follows. First, in Section 2, we introduce the structure of programs and the computational models considered in this paper. In Section 2.1, we present the transformation from shared memory model to WAC model. Then, in Section 3, we introduce the notion of slow and fast actions. Subsequently, in Section 4, we provide three illustrative examples. And, in Section 5, we analyze the effect of slow and fast actions. Section 6 presents the evaluation of the illustrative examples while in Section 7, we present an approach for slow-motion execution of fast actions. In Section 8, we discuss some of the questions raised by this work, and finally, in Sections 9 and 10, we explain related work and make concluding remarks. 2. Preliminaries A program p is specified by a set of variables Vp and a set of processes. Each process j of program p is described in terms of a set of guarded commands [11]. Each guarded command (respectively, action) is of the form guard −→ statement , where guard is a predicate over Vp , and statement updates the program variables. An action g −→ st is enabled when g evaluates to true and to execute that action, st is executed. A computation of this program consists of a sequence s0 , s1 , . . . , where sl+1 is obtained from sl (0 ≤ l) by executing actions (one or more, depending upon the semantics being used) in the program. Constraints on accessing remote variables in actions. A process can read variables of other processes while evaluating guards of its actions. The copies of these variables can be used in updating the process variables. Hence, we allow declaration of constants in the guard of an action. Intuitively, these constants save the value of the variable of the other process so that it can be used in the execution of the statement. As an illustration, consider a program where there are two processes j and k with variables x.j and x.k respectively. Hence, an action where j copies the value of x.k when x.j is less than x.k is specified as follows: Let y = x.k x.j < y −→ x.j = y.

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

43

Fig. 1. TDMA based transformation algorithm (see Refs. [5,6,21]).

The definition of action specified above allows a process to read the variables of all processes. Often, in a distributed program, for several reasons, it is necessary that a process can only read the variables of a small subset of processes. We use the term neighborhood to denote variables that a process can read. In other words, neighborhood of j consists of all the processes whose variables can be read by j. A computation model limits the variables that an action can read and write. We now describe how we model the restrictions imposed by shared memory model and WAC model. Shared memory model. In shared memory model, in one atomic step, a process can read its state as well as the state of all its neighbors and write its own (public and private) variables. However, it cannot write variables of other processes. Write all with collision (WAC) model. In WAC model, each process (or node) consists of write actions (to be precise, write-all actions). Specifically, in one atomic action, a process can update its own state and the state of all its neighbors. In this model, if two or more processes simultaneously try to update the state of another process, say l, then the state of l remains unchanged. Thus, the WAC model captures the fact that a message sent by a node is broadcast, and if multiple messages are sent to a node simultaneously then, due to collision, it receives none. 2.1. Basic transformation from shared memory model to WAC model Our transformation algorithm from SF shared memory model to WAC model is based on the algorithm in [22] that transforms a (regular) shared memory model into WAC model. Specifically, in [22], authors consider two transformation algorithms, one based on TDMA and one based on CSMA. While either of these could be extended to SF shared memory model, for brevity, we only

focus on TDMA based transformation. We discuss this further in Section 8. In this algorithm, each node maintains a copy of all (public) variables of its neighbors. These copies are used in all shared memory actions that need to access remote variables. Furthermore, node j updates its public variables only when it is allowed to execute (in its TDMA slot). And, when j executes, it broadcasts all its public variables so that the neighbors can update their copies. The algorithm for transformation is as shown in Fig. 1. From [22], we recall the following property of this algorithm: Theorem 1. Let p be the given program in shared memory model. And, let p′ be the corresponding program in WAC model transformed using the algorithm in Fig. 1. For every computation of p′ in WAC model there is an equivalent computation of p in shared memory model.  3. Slow and fast actions As discussed in Section 2.1, when a process executes its shared memory actions, it utilizes the copy of the neighbors’ state. However, when message losses occur, it is possible that the information j has is stale. In this section, we discuss how a node can determine whether it is safe to execute its action. For the following discussion, let g −→ st be a shared memory action ac at node j. To execute ac, j needs to read the state of some of its neighbors to evaluate g and then execute st if g evaluates to true. Let N denote the set of neighbors whose values need to be read to evaluate g. In the context of WSNs, j obtains its neighbors’ values by allowing the neighbors to write the state of j. In addition to the algorithm in Fig. 1, where each node writes its own state to its neighbors, we require the update to be associated with a timestamp which can be

44

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

implemented easily and efficiently.1 Next, we focus on how j can determine whether g evaluates to true. 3.1. When do we evaluate the guard? The first approach to evaluate g is to ensure that the knowledge j has about the state of nodes in N is up-to-date. Let Cur denote the current time and let tk denote the time when k notified j of the state of k. The information j has about nodes in N is latest iff for every node k in N, k was not assigned any TDMA timeslot between (tk , Cur). Definition 1 (Latest). We say that j has the latest information with respect to action ac iff latest (j, ac ) is true, where latest (j, ac ) = (∀k : k ∈ N : k updated the state of j at time tk and k does not have a TDMA slot in the interval (tk , Cur), where Cur denotes the current time.) Clearly, if latest (j, ac ) is true and g evaluates to true then g is true in the current global state, and, j can execute action ac. Of course, if action ac depends upon several neighbors then in the presence of message loss or sleeping nodes, it may not be easy for j to ensure that g holds true in the current state. For this reason, we change the algorithm in Fig. 1 as follows: Instead of maintaining just one copy for its neighbors, j maintains several copies with different time values (i.e., snapshots). Additionally, whenever a node updates its neighbors, instead of just including the current time, it includes an interval (t1 , t2 ) during which this value remains unchanged. Based on these, we define the notion that j has a consistent information about its neighbors although the information may not be most recent. Definition 2 (Consistent). We say that j has consistent information as far as action ac is concerned iff consistent (j, t , ac ) is true, where consistent (j, t , ac ) = (∀k : k ∈ N : k updated the state of j at time tk and k does not have a TDMA slot in the interval (tk , t )). Observe that if consistent (j, t , ac ) is true and g evaluates to true based on most up-to-date information at time t then this implies that it is safe to execute action ac at time t. After j executes it can throw out the old snapshots, and has to start collecting new snapshots. As we show in Section 5, at most 3 or 4 snapshots is enough for finding a consistent cut, so the memory overhead of storing old snapshots is low. Even though satisfying latest (j, ac ) may be difficult due to message losses and/or sleeping nodes, satisfying consistent (j, t , ac ) is easier (see Section 5). To observe that if j misses an update from its neighbor, say k, in one timeslot then j may be able to obtain that value in the next timeslot. Moreover, if state of k had not changed in the interim, j will be able to detect if a guard involving variables of k evaluates to true. Furthermore, if action ac involves several neighbors of j then it is straightforward to observe that the probability that consistent (j, t , ac ) is true for some t is significantly higher than the probability that latest (j, ac ) is true.

1 In the context of TDMA and the algorithm in Fig. 1, the timestamp information can be relative. Based on the results in Section 5, it would suffice if only 2–4 bits are maintained for this information.

The notion of consistency can be effectively used in conjunction with sleeping nodes. In particular, if node k is expected to sleep during an interval (t1 , t2 ), it can include this information when it updates the state of j. This will guarantee j that state of k will remain unchanged during the interval (t1 , t2 ) thereby making it more feasible to ensure that it can find a consistent state with respect to its neighbors. 3.2. When do we execute the action? The problem with the notion of consistency is that even though the guard of an action evaluated to true at some point in the past, it may no longer be true. To deal with this problem, we introduce the notion of a slow action and the notion of a fast action. We call the resulting shared memory model as SF shared memory model. Definition 3 (Slow Action). Let ac be an action of j of the form g −→ st. We say that ac is a slow action iff the following constraint is true:

(g evaluates true at time t ) ∧ (j does not execute any action between interval [t , t ′ ]) ⇒ (g evaluates true at time t ′ ). Rule 1: Rule for execution of a slow action. Let ac be a slow action of node j. Node j can execute ac provided there exists t such that consistent (j, t , ac ) is true and j has not executed any action in the interval [t , Cur) where Cur denotes the current time. Definition 4 (Fast Action). Let ac be an action of j of the form g −→ st. We say that ac is a fast action iff it is not a slow action. Rule 2: Rule for execution of a fast action. Let ac be a fast action of node j. Node j can execute ac provided latest (j, ac ) is true. If the algorithm in Fig. 1 is modified based on the above two rules, i.e., slow actions can be executed when their guard evaluates to true at some time in the past and fast actions are executed only if their guard evaluates to true in the current state, then we can prove the following theorems: Theorem 2. Let j and k be two neighboring nodes with actions ac 1 and ac 2 respectively. If both ac 1 and ac 2 are slow actions then their execution by Rule 1 is serializable.  Proof. This statement is trivially satisfied if ac 1 and ac 2 do not share variables. Hence, we consider the case where ac 1 and ac 2 are actions of neighboring processes j and k of the form g1 −→ st1 and g2 −→ st2 respectively. Let t1 be the time when ac 1 is executed, and let t2 be the time when ac 2 is executed. By TDMA, we know that t1 ̸= t2 . Without loss of generality, let t1 > t2 . Each process only updates its state in its TDMA slot. If consistent (j, t , ac 1 ) is true and j does not execute in time [t , t1 ] then g1 is still true at time t1 . Furthermore, if ac 1 is a slow action of process j and uses a variable of process k then based on the constraints of actions identified earlier and the definition of slow action, remote variables used in ac 1 remain unchanged in the interval [t , t1 ]. Hence, the effect of execution of ac 1 based on the values read at time t is identical to the effect of the execution of ac 1 based on the values at time t1 . In other words, the execution of the program is identical to a serial execution of ac 2 and then ac 1 .  Theorem 3. Let j and k be two neighboring nodes with actions ac 1 and ac 2 respectively. If both ac 1 and ac 2 are fast actions then their execution by Rule 2 is serializable.  Proof. This follows trivially from the fact that each process only executes in its TDMA slot. 

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

45

Fig. 2. Constructing tree program.

Theorem 4. Let j and k be two neighboring nodes with actions ac 1 and ac 2 respectively. Let ac 1 be a slow action and let ac 2 be a fast action. Then, their execution according to Rules 1 and 2 is serializable.  Proof. If ac 1 executes before ac 2 , we use the fact that each process only executes in its TDMA slot and, hence, their execution is serializable. If ac 2 executes before ac 1 , the proof would be similar to that of Theorem 2.  Summarizing SF shared memory model. To sum up, a SF shared memory program is a shared memory program where each action is annotated to be either a slow action or a fast action. If the action is annotated as a slow action then it satisfies the constraints of Definition 3. Computations of a SF shared memory program are derived in the same manner as ordinary shared memory programs. In other words, a computation of program p is of the form ⟨s0 , s1 , . . .⟩, where in each state, we identify one or more actions whose guards are enabled and execute the corresponding statement(s) atomically. SF shared memory model is only making it easier to check that guard of a slow action is true by observing that the guard remains true as long as the corresponding process does not execute. 4. Illustrative examples In this section, we present three algorithms to illustrate the notion of slow and fast actions. Of these, the first program (Section 4.1) is a program to reconstruct a spanning tree when existing nodes have failed and/or repaired. This program typically forms a backbone of routing and data propagation. The second program (Section 4.2) is a distributed reset program that is used to reset the program to a predefined initial state. Such a program is useful in reorienting the given network to perform new tasks as well as to recover the network from loss of coordination. Finally, the third program (Section 4.3) is an unison protocol that allows nodes to progress in a controlled manner so that neighboring nodes are in unison with each other. This protocol can be used to correlate observations at different nodes. 4.1. Tree construction program As the first algorithm, we use the tree construction program from [2]. In this tree construction program (cf. Fig. 2), each node j maintains three variables: P .j that denotes the parent of node j, root .j that denotes the ID of the node that j believes to be the root, and color .j that is either green (node believes that the tree is not broken) or red (node has evidence that the tree is broken). Each node j also maintains an auxiliary variable up.j that denotes whether j is up or whether j has failed. The protocol consists of five actions. The first three are program actions whereas the last two are environment actions that cause a node to fail and recover respectively. The first action allows a node

to detect that the tree that it is part of may be broken. In particular, if j finds that its parent has failed then it sets its color to red. This action also fires if there is a parent and the parent is colored red. Observe that with the execution of this action, if a node is red then it will eventually cause its descendants to be red. The second action allows a red node to separate from the current tree and form a tree by itself provided it has no children. The third action allows one node to join the tree of another node. In particular, if j observes that its neighbor k has a higher root value and both j and k are colored green then j can change its tree by changing P .j to k and root .j to root .k. The fourth action is a fault action that causes a node to fail (i.e., up.j = false). Due to the execution of this action, the first action will be enabled at the children. And, finally, the last action allows a node to recover. When a node recovers, it sets its color to red. We can make the following observations about this program. Theorem 5. AC 1t and AC 2t are slow actions. Proof. If a node detects that its parent has failed or its parent is red then this condition is stable until that node (child) separates from the tree by executing action AC 2t . Hence, AC 1t is a slow action. Likewise, if a node is red and has no children then it cannot acquire new children based on the guard of AC 3t .  Theorem 6. AC 3t is a fast action. Proof. After j evaluates its own guard for AC 3t , it is possible that the guard becomes false subsequently if k changes its color by executing AC 1t or if k changes its root by executing AC 3t . Hence, AC 3t is a fast action. 4.2. Distributed reset protocol To illustrate the notion of slow and fast actions further, in this section, we use the distributed reset program from [3]. The purpose of this program is to augment an arbitrary distributed system so that each of its processes can reset the system to a predefined global state, when deemed necessary. The protocol in [3] first builds a spanning tree. Then, it performs the reset operation on that tree. For simplicity of presentation, in this paper, we only focus on the reset operation. Hence, initially, we ignore the faults that cause the tree to be partitioned and actions that would fix the tree. The issue of tree construction can be handled by a protocol similar to that in Section 4.1. The distributed reset protocol can be used in sensor network in several ways. For example, it can be used to reorient the sensor network to perform a new task. In this case, it is important to initialize the state of the network for that new task. It could also be used to update systemwide parameters (e.g., threat level) in the network. After the tree is constructed, the reset program has three phases. In the first phase, one of the processes propagates a global

46

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

Fig. 3. Distributed reset program.

reset request along the spanning tree path toward the tree root. However, for simplicity, we assume that the tree root has already received the reset request. In the second phase, the tree root resets its state to a predefined global state and initiates a reset wave that propagates along the tree toward the leaves. Whenever the reset wave received by a process, say j, it resets its state to the predefined global state. In the last phase, after the reset reaches the tree leaves, it is reflected as a completion wave toward the tree root. The reset is complete when the completion wave reaches the tree root. In this program, each node j maintains three variables: P .j that denotes the parent of node j, st .j that shows whether j is in the middle of a reset operation (reset) or whether it has completed its last reset operation (normal), and sn.j that denotes a session number to distinguish between different reset waves. Starting the reset program from the tree root, the protocol consists of four actions. In the first action, the root initiates a reset wave. In the second action, it propagates the reset computation toward the leaves. In the third action, the reset computation completes from the children of a process to the process. The last action ensures the self-stabilization of the protocol by ensuring that no matter what the initial state is, the program can recover to legitimate states from where future reset operations work correctly (see Fig. 3). We can make the following observations about this program.

4.3. Asynchronous unison protocol As another example, we use asynchronous unison protocol from [10]. In an asynchronous unison protocol each node j has a variable x.j called clock, which is a positive integer. The node j checks the clock values of its neighbors in sequence. If the clock value of j is less/equal than/to that of its neighbor, say k, then j checks the clock of its next neighbor, and so on. After j has gone through each of its neighbors successfully, it increments its clock. In our evaluation, we assume that each node j in the asynchronous unison protocol has one action as follows (see Fig. 4): Such an unison protocol is an abstract version of clock synchronization where the clocks in a given neighborhood are kept close to each other. Thus, the asynchronous unison can be used to correlate observations at different nodes. Considering the protocol we can make the following observation. Theorem 9. AC 1u is a slow action. Proof. If guard of AC 1u is true at process j, and its neighbor k executes an action, x.k is increased. With this change, the guard of AC 1u at process j is still true. Hence, AC 1u is a slow action. 5. Analytical comparison of slow and fast actions

Theorem 7. AC 1r and AC 3r are slow actions. Proof. The guard of AC 1r is a local guard and, hence, it is not affected by execution of other actions. Regarding action AC 3r , we can observe that if the set of nodes in the network form a tree and all children of node j have completed the diffusing computation that j propagated to them then children of j cannot propagate any further diffusing computation until j completes its current diffusing computation. Hence, AC 3r is a slow action.  Remark. If we begin in states where tree reconfiguration may be necessary, we can still ensure that AC 3r is a slow action; specifically, when a node, say j changes its parent to k, we can require that j copies the sequence number of k and changes its state to normal. With this change, it is straightforward to see that AC 3r will continue to be slow action even if tree reconfiguration is allowed. Moreover, since AC 1r only involves actions of one process, it will still be a slow action even if tree reconfiguration is allowed. Theorem 8. AC 2r and AC 4r are fast actions. Proof. If guard of AC 2r is satisfied, it is possible that the parent of j (P .j) can change its state due to action AC 4r . Hence, AC 2r is a fast action. For the same reason, even AC 4r is a fast action.

In this section, we evaluate the conditions for executing slow and fast actions when the system is subjected to message loss. For the execution of a fast action, each node j needs to evaluate whether it has obtained the latest information about the state of its neighbors. If latest (j, ac ) evaluates to true for some action then j can evaluate the guard of that action and execute the corresponding statement. If latest (j, ac ) is false for all actions then j must execute a ‘skip’ operation and see if it can obtain the latest information in the next TDMA round. For the execution of a slow action, j proceeds in a similar fashion. However, if j obtains consistent information about its neighbors that is not necessarily from the latest TDMA round, j can execute its action. Next, we evaluate the probability that j can obtain the necessary consistent and/or latest state information. For this analysis, let p be the probability of a message loss and let N denote the number of neighbors whose status needs to be known to evaluate the guard of the action. If j cannot successfully obtain consistent and/or latest state information in one TDMA round then it tries to do that in the next round. Hence, we let m denote the number of TDMA rounds that j tries to obtain the consistent and/or latest information. Assuming that states of the neighbors do not change during these m

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

47

Fig. 4. Asynchronous unison protocol.

(a) Latest probabilities.

(b) Consistent probabilities. Fig. 5. Latest and consistent probabilities for p = 10%.

(a) Latest probabilities.

(b) Consistent probabilities. Fig. 6. Latest and consistent probabilities for p = 20%.

rounds, we calculate the probability that j can obtain the required consistent and/or latest state information.2 Probability of obtaining latest information. To obtain the latest information in one TDMA round, j needs to successfully receive message from each of its neighbors. Probability of successfully receiving message from one neighbor is (1 − p). Hence, the probability of obtaining latest information in one round is (1 − p)N . And, the probability of not obtaining the latest information in one round is 1 − (1 − p)N . Therefore, probability of not obtaining the latest information in any one of m rounds is (1 − (1 − p)N )m . Thus, the probability that j can obtain the latest information in at least one m rounds is (1 − (1 − (1 − p)N )m ). Probability of obtaining consistent information. To obtain consistent information in the earliest of m rounds, j needs to obtain information from each of its neighbors in some round.

2 We can relax this assumption by requiring the nodes to include their old values in previous rounds with their broadcast. These values are then used for finding a consistent cut in the past. Our results in this section show that it suffices for the node to include values from the last 3 rounds for most cases. Observe that this method does not help ‘‘latest’’ because learning an older snapshot does not allow executing a fast action.

(Observe that since the nodes include the intervals where their value is unchanged, receiving a message from each node at some round is enough for identifying the first round as the consistent cut.) The probability that j does not receive message from one of its neighbors in either of m rounds is pm . Hence, probability of successfully receiving message from one neighbor is (1 − pm ). Therefore, probability of successfully receiving message from every neighbor is ((1 − pm ))N . Furthermore, there is an additional conditional probability where j fails to get consistent information in the first (earliest) round but obtains it in the next round. We take this into account in our calculations and graphs, but omit the full formula here for the sake of brevity. Next, we compare the probability of obtaining the latest and consistent information for different values of p, N and m. Figs. 5–7 show the latest and consistent probabilities for p = 10%, p = 20%, and p = 30% respectively. First, we note that the probability of obtaining latest information decreases as N increases for different values of m. A given node has to receive updates from all its neighbors in order to obtain the latest information. Hence, as N increases, latest probability decreases. Moreover, in a high message loss environment (e.g., p = 20% and p = 30%), latest probabilities decrease significantly as N increases. For small neighborhoods, the probability of getting latest information improves as m increases.

48

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

(a) Latest probabilities.

(b) Consistent probabilities. Fig. 7. Latest and consistent probabilities for p = 30%.

This suggests that if the neighbors remain silent (i.e., do not change state) for some rounds then the probability of obtaining latest information improves. On the other hand, although the probability of obtaining consistent information decreases as N increases, for m ≥ 3, it remains close to 1. Therefore, obtaining a consistent cut has a higher probability compared to obtaining latest state. The graphs show that by choosing m = 3 the probability of finding a consistent cut is virtually certain at 10% message loss rate. The above discussion shows that the probability of obtaining the consistent information is significantly higher than that of latest information. In turn, this suggests that it is better to utilize protocols that have slow actions versus protocols that have fast actions. In particular, it is better if actions that depend on the value of several neighbors are slow actions. On the other hand, if protocols must have fast actions, then it is better if they rely on a small number (preferably 1) of neighbors. 6. Simulation results In this section, we evaluate the examples of Sections 4.1, 4.2, and 4.3 when the system is subjected to message loss. Specifically, we evaluate the effectiveness of slow actions in improving the performance of these examples. Recall that in Sections 4.1, 4.2, and 4.3, we have identified slow actions for the respective protocols. Moreover, correctness of the protocol is not affected if these slow actions are executed as fast actions. However, performance may be affected in this situation. Hence, we consider two scenarios, one where all actions are executed as fast actions and another where the knowledge of slow and fast actions is utilized. Since the programs in Sections 4.1, 4.2, and 4.3 utilize guarded commands, we use ProSe [7], which is a programming tool for sensor networks, to transform the programs to nesC/TinyOS platform [15]. Then, using TOSSIM [25], we simulate the generated programs and evaluate the effect of having slow actions in each program. In the rest of this section, first, in Section 6.1, we explain ProSe and its input/output. Afterwards, in Sections 6.2, 6.3, and 6.4, we present the simulation results and describe the effectiveness of slow actions in improving the performance. 6.1. ProSe ProSe (Programming tool for rapid prototyping of Sensor networks) is a programming platform for sensor networks that allows designers to concisely specify sensor network protocols. ProSe enables the designers to (1) specify protocols and macro-programming primitives in simple, abstract models (e.g.,

read/write model, shared-memory model), (2) transform the programs into WAC model while preserving properties such as faulttolerance and self-stabilization [11] of the original programs, and (3) automatically generate and deploy code. An advantage of ProSe is that it facilitates the designer to reuse existing faulttolerant/self-stabilizing protocols from the literature in the context of sensor networks. Moreover, since the programmer only specifies the abstract protocol, ProSe helps to overcome deficiencies of existing event-driven programming platforms (e.g., nesC/TinyOS) that require the programmers to deal with several challenges, including, buffer management, stack management, and flow control [7]. The input of ProSe is a guarded command program, which is explained in Section 2. However, its syntax is a little different from the programs explained in Sections 4.1, 4.2, and 4.3. For instance, consider the first three actions in Fig. 2. The input of ProSe for these actions in the shared memory model is as follows: ((color.j == 1) && (([P.j].up == FALSE) || ([P.j].color == 0))) -> color.j = 0; | ((color.j == 0) && (forall k : k in Nbr.j : ([k].P != j))) -> color.j = 1; P.j = j; root.j = j; | ((root.j < [k].root) && (color.j == 1) && ([k].color == 1)) -> P.j = k; root.j = [k].root;

The output of ProSe is a program in nesC/TinyOS platform. To evaluate the effectiveness of slow actions, we extended ProSe so that each action is marked as either a slow action or a fast action. In the implementation of ProSe to generate the nesC program, ProSe maintains copies of variables of its neighbors. These copies are updated when a node executes. Specifically, when node j executes, it updates the local copies of its own state maintained by the neighbors. These copies are timestamped with the timeslot of when they are updated. If a node fails to update an information, e.g., due to message loss, the recipient marks this copy as unavailable. Moreover, when j needs to read the state of another node, it utilizes this copy variable. Note that as discussed earlier, it is possible that the copies of variables of different processes are not consistent with each other. Our extension allows ProSe to determine if these copies can be utilized to obtain the latest information (cf. Definition 1) or to obtain a consistent information (cf. Definition 2). In the former case, either a fast or a slow action can be executed, whereas in the latter case, only a slow action can be executed. The maximum number of copies maintained by ProSe is determined by a given parameter to ProSe; the more copies that are maintained the greater the probability for obtaining consistent (although possibly older) information at the increased storage cost.

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

6.2. Effectiveness of slow actions under message loss In all the evaluations of this section and Sections 6.3 and 6.4, we use the following experimental setup. All sensor nodes are communicating in TDMA mode and deployed in grid. In the applied network, the number of sensors is either 25, 36, 49, 64 or 100, and to simulate unreliable links, different levels of message loss rate are introduced from 10% to 50%. We also utilize the empirical radio model, which models the real world situation as it implements attenuation of transmitting with signals based on Gaussian distribution. As we mentioned in Section 4.1, the tree construction program of Fig. 2 has three program actions. Since AC 3t is a fast action and we need the latest information to execute it, we do not use the older copies for this action. However, since AC 1t and AC 2t are slow actions, we use the older copies to execute them. To illustrate the effect of using older copies for these slow actions (AC 1t and AC 2t ), we evaluate the nesC program of the tree construction program by considering different numbers of sensor nodes and different message loss rates while recovering the tree from a node failure. Since the exact identity of the failed node can affect the time required to reconstruct the tree, we consider the case where in the initial state, a consistent (and identical) tree is formed. Moreover, in this state, we set the root of the tree to have failed. Once the failure of the root is detected, all its children would become red and recursively, the grandchildren would be colored red as well. After that, nodes rebuild the new tree that excludes failed nodes. We measure how quickly slow actions and fast actions accomplish this recovery process respectively. Fig. 8 shows the time of tree recovery after node failure with 10%, 20%, 30%, 40%, and 50% message loss in the network. In the following figures, F, short for Fast mode, means the program only uses fast mode, and SF, short for Slow–Fast mode, means the program applies both slow and fast actions. For example, F-0.5 means we have 50% message loss while having just fast actions, and SF-0.1 means we have 10% message loss while having both slow and fast actions in the network. In Fig. 8, we see that as the number of neighbors increases the recovery time of the tree increases in both cases (either fast or slow/fast). The reason is that a given node has to receive updates from all of its neighbors in order to obtain the latest information. Moreover, comparing the tree recovery time of both cases, the time is less when we have slow actions in the program. In all these five figures the one using slow actions outperforms and improves the recovery time by more than 20% compared to the one using only fast actions. Also, in a high message loss environment (e.g., 40% and 50%), having slow actions helps more in decreasing the recovery time (see Fig. 8(d) and (e)). This is because the probability of having consistent information increases as we use slow actions. In other words, if the neighbors remain silent for some rounds, the sensor can use consistent information. However, fast actions are not executed if the sensor does not receive the latest information. Thus, we find that the slow actions improve the performance of the protocol. Next, we evaluate the effect of having slow actions on the distributed reset program explained in Section 4.2. In this evaluation, we assume that we have a pre-built tree and evaluate the time needed for the distributed reset in the tree while having different sensor node numbers and message loss rates. The reason for this is that the program in Section 4.2 does not have actions to build a tree. Actions such as that in 4.1 can be used for this purpose. Fig. 9 shows the time needed for completing the reset with 10%, 20%, 30%, 40%, and 50% message loss in the network. Increased message loss increases the time for performing distributed reset. However, in all the cases, slow actions significantly reduce the time for completing the reset. For instance, having 10% message loss, the one having slow actions saves 10%–15% of the time compared

49

to the one having only fast actions. In the scenario where we have 30%–40% message loss, using slow actions improves the reset time almost 40% while having 50% message loss improves the reset time more than 50%. In fact, the effectiveness of slow actions is higher in a high message loss environment. Once again, this is consistent with the theoretical analysis from Section 5. 6.3. Effectiveness of slow actions in successful execution of actions In this section, we evaluate the effect of slow and fast actions in terms of the ability to execute the corresponding action. In order to evaluate the ability of executing different kinds of actions, we measure the number of TDMA rounds when the guards of actions are true for a node, say j. Then, we repeat this measurement for all the nodes in the network to calculate the summation of obtained numbers. Next, we use this summation to evaluate the percentage of rounds when the guards of actions are true. This percentage indicates the effectiveness of slow actions in successful execution of actions and is calculated as follows: Pse =

# of action executed successfully # of guard evaluations to determine feasibility of execution N 

=

Sj

j =1

(T ) ∗ (N )

where Sj is the number of TDMA rounds when the guards of actions are true for each node j, T is the total number of rounds, and N shows the total number of nodes in the network. Thus, when the value of Pse is higher, the sensor nodes have a better chance to run the actions successfully. Considering AC 3r as a slow action in the distributed reset program, Fig. 10 indicates the effect of message loss on Pse while increasing N from 25 to 100. As the figure illustrates, when AC 3r is a slow action, the number of rounds when the guard of AC 3r is true increases and as a result the amount of Pse improves. For example, having 10%–30% message loss (Fig. 10(a)–(c)), the improvement of Pse is 8%–66%. This improvement is even more considerable when we have higher message loss rate in the network. For instance, for 40%–50% message loss (Fig. 10(d) and (e)), the improvement is 65%–150%. Hence, we can conclude that, in a network with high message loss rate, which is more probable in wireless sensor networks, having slow actions improves the probability of execution of program actions. Note that the Pse  curves are descending since the rate of (T ∗ N) grows faster than Sj . Moreover, the simulation results for the tree construction and the asynchronous unison protocols apply the same conclusion. In the tree construction program, having AC 1t and AC 2t as slow actions increases the possibility of their execution and, hence, a greater amount of Pse . This is also the same when we consider AC 1u as a slow action in the asynchronous unison protocol. 6.4. Effectiveness of slow actions under different density conditions In this section, we analyze the execution conditions of slow and fast actions when the system is subjected to message loss in the asynchronous unison protocol. The purpose of this example is to have a single action program whose results are more realistic to be compared with the results of Section 5, since the example explained in Section 5 has one action as well. Compared to the other two programs whose slow–fast mode includes both slow actions and fast actions, the asynchronous unison protocol could get a more constructive result by purely comparing its action as a slow action versus a fast action. Using TOSSIM with the same setting like that in Sections 6.2 and 6.3, Fig. 11 shows the time needed for executing the single

50

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

(a) 10% message loss.

(b) 20% message loss.

(c) 30% message loss.

(d) 40% message loss.

(e) 50% message loss. Fig. 8. Different message loss rates while having either fast actions or both slow and fast actions in the tree construction program.

action of the asynchronous unison program. In this figure, each node has either 4, 6, 8, 10, or 12 neighbors. As the number of neighbors increases, the time needed for executing the single action gets larger. The time also increases when the message loss rate grows from 10% to 30%. Having a slow action, the time for executing the action is always better than having just a fast action in the program. In order to evaluate the probability of obtaining latest information, we use the same formula that we used in Section 6.3 to calculate Pse when AC 1u is considered as a slow action. Recall that, a node, say j, has the latest information with respect to action ac iff j

has all the updated information of its neighbors at time t and none of the neighbors has a TDMA round since time t. Therefore, in the asynchronous unison protocol, when the guard is true and none of the neighbors of j executes AC 1u , latest (j, AC 1u ) is true. Fig. 12(a) illustrates the latest probabilities while each node has either 4, 6, 8, 10, or 12 neighbors. We note that, as the number of neighbors increases, the probability of obtaining latest information decreases. The reason is that a node has to receive updates from more neighbors in order to obtain the latest information. We can also consider that in higher message loss probabilities, like 30% (F-0.3),

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

(a) 10% message loss.

(b) 20% message loss.

(c) 30% message loss.

(d) 40% message loss.

51

(e) 50% message loss. Fig. 9. Different message loss rate while having either fast actions or both slow and fast actions in the distributed reset program.

the probability of obtaining latest information decreases significantly as the number of neighbors increases. Also note that, the results are similar to the results of Figs. 5–7 while we have 10%–30% message loss and m = 4 (m denotes the number of TDMA rounds during which j tries to obtain the consistent/latest information). Also, the number of neighbors is either 4, 6, 8, 10, or 12. Furthermore, we measure Pse when AC 1u is considered as a slow action to calculate consistent information. Recall that node j has consistent information with respect to action ac and time t iff j has all the updated information till time tk and none of its neighbors has a TDMA slot between time tk and t. In Fig. 12(b), as the number

of neighbors increases, the probability of obtaining the consistent information decreases. These results are similar to that of Figs. 5– 7. More importantly, we consider that the probability of obtaining the consistent information is significantly higher than that of latest information. It validates the suggestion of utilizing protocols that have slow actions versus protocols that have fast actions. 7. Pseudo-slow actions The results in Section 5 show that if actions of a program are slow then their execution is expected to be more successful. Thus,

52

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

(a) 10% message loss.

(b) 20% message loss.

(c) 30% message loss.

(d) 40% message loss.

(e) 50% message loss. Fig. 10. Effectiveness of slow actions in successful execution of distributed reset actions. Each figure presents the percentage of guards that are true in different message loss rates.

the natural question is what happens if all program actions were fast? Can we allow such a program to utilize an old consistent state to evaluate its guard. We show that for a subset of the original actions, this is feasible if we analyze the original shared memory program to identify dependent actions. We illustrate our approach in the context of the tree example in Section 4.1. For sake of discussion, let us assume that all actions are fast actions; this is reasonable since it adds more restrictions on how each action can be executed. Furthermore, let us consider the case that we want j to be able to execute AC 3t by utilizing

a consistent state although not necessarily the latest state. Recall that action AC 3t causes j to join the tree of k. If j is using a consistent state that is not necessarily the latest state, it is possible that k has changed its state in the interim. Observe that if k had increased the value of root .k by executing AC 3t then it is still safe for j to execute action AC 3t . However, if k executes AC 1t and changes its color to red, subsequently observes that it has no children and executes AC 2t then it may not be safe for j to join the tree of k. Thus, if we want to allow j to execute AC 3t using a consistent state that is not necessarily latest then k must be prevented from executing either

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

53

if A and its dependent actions are disjoint. If there is an overlap between these two sets then the set of pseudo-slow actions needs to be revised until this condition is met. Step 3: Choosing the delay value. The next step is to identify how much old information can be used in evaluating the guard of an action. Essentially, this corresponds to the choice of x in the above example. We denote this as the delay value of the corresponding action. The delay value chosen for efficient implementation of pseudo-slow actions is also user dependent. The value will generally depend upon the number of neighbors involved in the execution of the pseudo-slow action. Based on the analysis from Section 5, we expect that a value of 3–4 is expected to be sufficient for this purpose. Let this delay value be denoted by x.

Fig. 11. Time needed for executing asynchronous unison action while having 10%, 20%, and 30% message loss.

AC 1t or AC 2t . Again, for the sake of discussion, let us assume that we want to restrict k from executing AC 2t . Hence, in this case, we will say that pseudo-slow execution of AC 3t .j is dependent upon slowing down AC 2t .k. In this approach, we allow j to utilize consistent snapshots for up to x previous TDMA rounds (i.e., j can execute AC 3t .j if it obtains a consistent state that is no more than x rounds before the current time and evaluates that the guard of AC 3t is true). However, in this case, if k ever wants to execute action AC 2t then it must stay silent for at least x + 1 TDMA rounds before executing action AC 2t . Note that this will essentially disable execution of action AC 3t .j (i.e., at the end of x + 1 silent rounds k knows that j cannot simultaneously execute AC 3t .j and interfere with the execution of AC 2t .k).3 We generalize this approach in terms of the following 4-step algorithm. Step 1: Identify pseudo-slow actions. First, the designer needs to identify the set of actions, A, that are fast actions but it is desired that they can execute as slow actions, where a node can utilize consistent (but not necessarily the latest) information about the state of neighbors. The choice of A is application dependent, i.e., it is based on the designers’ belief/observation that quick execution of these actions is likely to help execution of the program. We denote the actions in A as a set of pseudo-slow actions since they are not slow actions but behave similar to the slow actions. Step 2: Identify dependent actions. Let Aj be one of the pseudoslow actions in A that is to be executed by node j. Let Aj be of the form g −→ st. Since Aj is a fast action, this implies that if the guard of Aj is true in some state then it can become false by execution of actions of one or more neighbors of j. Hence, the goal of this step is to identify the set of actions, say A, such that if (1) g evaluates to true in some state, (2) no action from A is executed, and (3) no action of j is executed then it is still acceptable to execute the statement st in the given shared memory program. The value obtained for A is called the dependent actions of Aj . In this step, for each action in A, we identify the corresponding set of dependent actions. The dependent actions for A is obtained by taking the union of these dependent actions. Step 2 is successful

3 We can relax this x + 1 silent rounds requirement. For this, we modify the algorithm in Fig. 1 slightly where a node, say j, not only notifies its neighbors about its own state but also includes a timestamp information about messages received from its neighbors. With this change, k can either execute its action AC 2t if it stops transmitting for x + 1 rounds or if it checks that j is aware of its color being red and, hence, will not execute action AC 3t .j.

Step 4: Revising the transformation algorithm. The last step of the algorithm is to utilize A identified in Step 1, the corresponding dependent actions identified in Step 2 and the delay value identified in Step 3 to revise the transformation algorithm. In particular, we allow the pseudo slow action at j to execute if (1) j obtains consistent state information about its neighbors, (2) j does not have more recent information about its neighbors than the one it uses, and (3) no more than x TDMA rounds have passed since obtaining the consistent state information about neighbors. Additionally, a dependent action at j can execute if j does not transmit its own state for at least x + 1 rounds. (It is also possible for j to execute a dependent action earlier based on the knowledge j got about the state of its neighbors. However, for reasons of space, we omit the details.) 8. Discussion What is specific to write-all in our transformation algorithm? Why is this transformation not applicable for message passing? Write-all-with-collision (i.e., wireless broadcast) model helps a lot for our transformation, but is not strictly necessary. Our transformation is also applicable for message-passing, if on execution of an action at k at its TDMA slot, its state is made available to all of its neighbors before the next slot starts. It may not be easy and inexpensive to guarantee this condition for message passing, whereas for write-all with TDMA this condition is easily and inexpensively satisfied. Can we relax the TDMA communication assumption? The definitions of ‘‘latest’’ and ‘‘consistent’’ depend on the assumption that ‘‘k does not have another (missed) TDMA slot until the cut’’. This is the only place TDMA assumption is used, and this is used for ensuring that k does not execute any action in that interval, so k’s new state is not out of sync with the cached state in the cut. Without using TDMA, the same condition can be achieved by using an alternative mechanism to communicate that k will not update its state for a certain duration. For example k can include a promise in its message that it will not update its state for some interval (e.g., until its next scheduled update, or until its sleep period is over). Given that our transformation can tolerate message losses in the concrete model, dropping the TDMA mechanism would not hurt the performance of the transformed program significantly. The round concept could be used without the TDMA slots, and the nodes would utilize CSMA to broadcast their messages. What are the rules of thumb for marking actions as slow? As we mentioned in the Introduction, the protocol designer can mark an action as slow only if (1) guard is a stable predicate, (2) guard depends only on local variables (this covers a rich set of programs), or (3) guard is a ‘‘locally stable’’ predicate. While the first two conditions are easy to detect, the locally stable condition requires reasoning about the program execution. We expect the protocol designer to understand his program.

54

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

(a) Latest probabilities.

(b) Consistent probabilities. Fig. 12. Latest and consistent information for 10%, 20%, and 30% message loss rates.

A big problem is marking a fast action as slow, as this would violate correctness! It is better to err on the side of safety and mark the action as fast if there is some doubt about it being a slow action. Marking a slow action as fast does not violate correctness, but would just reduce the performance in the presence of message losses. Do we need to use slow-motion execution for every program? If the designer can mark all program actions as slow, there is obviously no need for slow-motion execution as there is no fast action remaining. Even when there are some fast actions remaining, if most of the actions are slow actions and message loss rates are not very high, these fast actions may not reduce the performance of the program significantly. However, if message loss rates increase further, it could be more beneficial to switch to slowmotion execution than to suffer from message losses voiding the latest cut and blocking the fast actions.

processing and control. This model views the network as a distributed platform for in-network processing. Furthermore, in this model, the abstraction of collaboration groups hides the designer from issues such as communication protocols, event handling, etc. Similarly, in [23], a state centric programming model called uDSSP is proposed where an application is composed of several state centric services. Each service interact with other services by subscribing to their states. Unlike [23,26], the SF shared memory model proposed in this paper enables the designer to evaluate existing algorithms in the context of sensor networks. Moreover, since the programs are written in abstract models considered in distributed systems, it is straightforward to verify the correctness of the programs as well as to manipulate the programs to meet new properties. For example, using [13], fault-tolerance properties can be automatically added to the programs specified in shared memory model.

Programming approaches proposed in [4,8,17,18,23,26,31–36] fall under the category of abstractions based on language aspects.

Macroprogramming. In [31,35], macroprogramming primitives that abstract communication, data sharing and gathering operations are proposed. These primitives are exposed in a highlevel language. However, these primitives are application-specific (e.g., abstract regions for tracking and gathering [35] and region streams for aggregation [31]). And, in [36], semantic services programming model is proposed where each service provides semantic interpretation of the raw sensor data or data provided by other semantic services. In this model, users only specify the end goal on what semantic data to collect. Thus, users make less low-level decisions on which operations to run or which data to run them over. In [17], macroprogramming model, called Kairos, that hides the details of code-generation and instantiation, data management, and control is proposed. Kairos provides the designers with three abstractions: (i) node-level abstraction that allows the designers to manipulate nodes and list of nodes, (ii) one-hop neighbor list abstraction for performing operations on the neighbor list, and (iii) remote data access that allows a sensor to read the named sensors. While the SF shared memory model provides similar abstractions, it differs from the programming approach proposed in Kairos. Specifically, SF shared memory model hides low-level details such as message collisions, corruption, sensor failures, etc. Additionally, unlike Kairos, SF shared memory model enables reuse of existing algorithms while preserving properties such as self-stabilization of the input program.

State centric approaches. In [26], a state centric approach is proposed that captures algorithms such as sensor fusion, signal

Rule based programming approaches. In [4,32,34], rule based programming approaches are proposed. These approaches allow

9. Related work In this section, we compare the SF shared memory model introduced in this paper with programming models and approaches proposed for sensor networks. In this discussion, we follow the categorization of programming approaches for sensor networks highlighted in [29]. In [29], high-level language constructs that allow programmers to express distributed processing among sensors are surveyed and classified. Prior to this study, programming approaches for sensor networks are categorized into node-centric programming and macroprogramming [17]. Nodecentric programming refers to programming abstractions and techniques from the point of view of individual nodes. Whereas, macroprogramming approaches consider overall behavior of the network. Unlike this categorization, [29] presents the taxonomy of programming approaches on two aspects: (1) language aspects and (2) architecture aspects. The taxonomy of language aspects deals with the primitives provided by different approaches and the programming models. And, the taxonomy of the architecture aspects deals with features such as their intended use and execution environment. 9.1. Programming abstractions based on language aspects

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

designers to specify programs similar to guarded commands format. However, unlike SF shared memory model, approaches proposed in [32,34] require designers to explicitly specify send/receive message actions of the sensors. As a result, the designers have to decide what messages to transmit (e.g., raw data vs. some interpretation of data), when message transmissions are scheduled (e.g., backoff based vs. timeslot based), and when to listen to the medium for new messages (e.g., always on radio vs. schedule based). Furthermore, the approaches proposed in [4,32,34] do not facilitate the reuse of abstract protocols from the literature. Moreover, dynamic embedded sensing and actuation language (DESAL) proposed in [4] do not provide a mechanism for preserving properties of interest (e.g., fault-tolerance, self-stabilization) in the transformed programs. In [33], a declarative sensor network programming paradigm called DSN is proposed. DSN uses Snlog, a high-level specification language based on facts and rules, for specifying programs. DSN provides an easy mechanism for interacting with the lower layers of the stack and components written in systems languages. On the contrary, SF shared memory model does not have to deal with the programming platforms and protocol stacks for sensor networks. One of the important differentiators between SF shared memory model and DSN is that properties of interest of given program is preserved during the transformation to WAC model. For example, if the given program is self-stabilizing then the transformed program is also self-stabilizing. Another significant difference is that DSN uses a query processor runtime to execute the rule-based constructs in the sensors. Whereas, the transformed program in WAC model does not assume any runtime support. In [18], a deductive framework for specifying sensor network programs are proposed. This approach views the network as a distributed database of facts gathered from the environment. As a result, the overall functionality of the network can be represented using deductive or logic rules. Non-collaborative functionality is provided as built-in functions. Unlike [18], the SF shared memory model relies only on the state of the neighbors. The overall collaborative functionality can be built based on the state of the neighbors. Moreover, SF shared memory model can be used to specify the built-in functions of [18] easily. In addition, the SF shared memory model enables reuse of existing built-in functions proposed in the literature. Virtual node abstraction. In [8], virtual node abstraction is proposed where the physical nodes in the network emulate the virtual node application (specified by the designer). The task of such emulation is divided among three main components: (1) to elect a region leader in each region of the network, (2) to retrieve the current state of virtual node application, and (3) to keep the virtual node state synchronized with the physical nodes in the region. This approach is similar to macroprogramming primitives. With SF shared memory model, the designers can reuse existing algorithms or specify new algorithms for leader election and state synchronization. Unlike SF shared memory model, the virtual node abstraction approach lacks the ability to reuse existing literature. Moreover, this approach requires a runtime support for emulating the virtual node. 9.2. Programming approaches based on architectural aspects Techniques like virtual machine (e.g., Maté [24]), middleware (e.g., EnviroTrack [1]), library (e.g., SNACK [16], TASK [9]), database (e.g., TinyDB [27]), and mobile agent (e.g., [14]) are proposed for simplifying programming sensor network applications. However, these solutions are (i) application-specific, (ii) architecture dependent, and/or (iii) restrict the designer to what is available in the virtual machine, middleware, library, or network. By contrast, the

55

SF shared memory model provides a simple abstraction while allowing the designer to specify wide variety of protocols.

10. Conclusion In this paper, we presented a new shared memory model called SF shared memory model. This model partitioned the actions into slow actions and fast actions. A slow action is one such that once it is enabled at a node j, it can be executed at any later point at j provided that j does not execute another action in between. Slow actions mean that the process can tolerate slightly stale state from other processes, which enables the concrete system to be more loosely-coupled, and tolerate communication problems better. We presented an algorithm for transforming a program from SF shared memory model to WAC model [20] so that it can be implemented in sensor networks. This algorithm enables the designer to utilize existing shared memory programs for their application in sensor networks. We quantified the improvements possible by using a slow action, and gave practical rules that help a programmer to mark his program actions as slow and fast. We also analyzed the feasibility of execution of slow and fast actions in scenarios with different varying losses and different neighbors. Toward this end, we defined the notion of consistent state and latest state. A consistent state is sufficient to ensure that execution of a slow action is correct. By contrast, a fast action required the latest state. We also showed that the probability of obtaining a consistent state is significantly higher than that of obtaining the latest state. Hence, probability of successful execution of a slow action is also high. Moreover, we showed that with small overhead (in terms of memory), it is possible to ensure that obtaining a consistent state is almost equal to 1. We illustrated the SF shared memory model with three programs, a tree construction program, distributed reset program and asynchronous unison protocol. In each program, we explained how to divide the program actions into slow and fast actions. We also evaluated these three programs under different density conditions and different loss rates. We showed that the use of slow actions improves the performance by 6%–47% for the tree program, 8%–61% for the distributed reset program and 21%–88% for the asynchronous unison program. Moreover, the improvement is higher for higher loss rate. And, the improvement is also higher when there is higher density. For reducing the performance penalty of fast actions under heavy message loss environments, we also introduced the notion of slow-motion execution for fast actions. This allows designers a tradeoff where they can speedup certain actions at the cost of other actions. One of the future work in this area is to develop guidelines so that the designer can identify when such approach would improve performance. In future work, we also plan to investigate adaptive switching to slow-motion execution to curb the performance penalty that message losses incur on fast actions. To this end, we will determine the break-even point for switching to the slow-motion execution mode, and middleware for switching to and back from the slowmotion mode seamlessly. We will also investigate the possibility of division of actions in any distributes system to slow and fast actions. It will help the system to have a better performance while the system is in a cloud, since possibility of execution of slow actions is higher in the systems subjected to high message loss, which is very probable nowadays that everyone uses smart phones.

56

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57

Appendix. Notion

Symbol WAC model SF shared memory model ac AC 1t . . . AC 5t AC 1r . . . AC 4r AC 1u

Description Write all with collision model Slow–fast shared memory model Arbitrary program action Specific program actions in tree coloring program Specific program actions in distributed reset program Specific program action in asynchronous unison program

References [1] T. Abdelzaher, B. Blum, Q. Cao, Y. Chen, D. Evans, J. George, S. George, L. Gu, T. He, S. Krishnamurthy, L. Luo, S. Son, J. Stankovic, R. Stoleru, A. Wood, EnviroTrack: Towards an environmental computing paradigm for distributed sensor networks, in: Proceedings of the International Conference on Distributed Computing Systems, ICDCS, March 2004. [2] A. Arora, Efficient reconfiguration of trees: A case study in the methodical design of nonmasking fault-tolerance, Sci. Comput. Program. (1996). [3] A. Arora, M. Gouda, Distributed reset, IEEE Trans. Comput. 43 (9) (1994) 1026–1038. [4] A. Arora, M. Gouda, J. Hallstrom, T. Herman, B. Leal, N. Sridhar, A state-based language for sensor–actuator networks, in: Proceedings of the International Workshop on Wireless Sensor Network Architecture, April 2007. [5] M. Arumugam, A distributed and deterministic TDMA algorithm for write-allwith-collision model, in: Proceedings of the 10th International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS), in: LNCS, vol. 5340, 2008. [6] M. Arumugam, S.S. Kulkarni, Self-stabilizing deterministic time division multiple access for sensor networks, AIAA J. Aerosp. Comput. Inf. Commun. (JACIC) 3 (2006) 403–419. [7] M. Arumugam, S. Kulkarni, ProSe: A programming tool for rapid prototyping of sensor networks, in: Proceedings of the First International Conference on Sensor Systems and Software, S-Cube, 2009, pp. 158–173. [8] M. Brown, S. Gilbert, N. Lynch, C. Newport, T. Nolte, M. Spindel, The virtual node layer: A programming abstraction for wireless sensor networks, in: Proceedings of the International Workshop on Wireless Sensor Network Architecture, April 2007. [9] P. Buonadonna, D. Gay, J. Hellerstein, W. Hong, S. Madden, TASK: Sensor network in a box, in: Proceedings of the European Workshop on Wireless Sensor Networks, EWSN, 2005, pp. 133–144. [10] J. Couvreur, N. Francez, M. Gouda, Asynchronous unison, in: Proceedings of the International Conference on Distributed Computing Systems, 1992, pp. 486–493. [11] E.W. Dijkstra, Self-stabilizing systems in spite of distributed control, Commun. ACM 17 (11) (1974). [12] S. Dolev, A. Israeli, S. Moran, Self-stabilization of dynamic systems assuming only read/write atomicity, Distrib. Comput. 7 (1993) 3–16. [13] A. Ebnenasir, S.S. Kulkarni, A. Arora, FTSyn: A framework for automatic synthesis of fault-tolerance, Int. J. Softw. Tools Technol. Trans. 10 (5) (2008) 455–471. [14] C.-L. Fok, G.-C. Roman, C. Lu, Rapid development and flexible deployment of adaptive wireless sensor network applications, in: Proceedings of the International Conference on Distributed Computing Systems, ICDCS, June 2005. [15] D. Gay, P. Levis, R. von Behren, M. Welsh, E. Brewer, D. Culler, The nesC language: A holistic approach to networked embedded systems, in: Proceedings of Programming Language Design and Implementation, PLDI, June 2003. [16] B. Greenstein, E. Kohler, D. Estrin, A sensor network application construction kit (SNACK), in: Proceedings of the Second ACM Conference on Embedded Networked Sensing Systems, SenSys, November 2004. [17] R. Gummadi, O. Gnawali, R. Govindan, Macro-programming wireless sensor networks using Kairos, in: Proceedings of the International Confernece on Distributed Computing in Sensor Systems, DCOSS, 2005. [18] H. Gupta, X. Zhu, Deductive framework for programming sensor networks, in: International Conference on Data Engineering, ICDE, 2009. [19] T. Herman, Models of self-stabilization and sensor networks, in: Proceedings of the 5th International Workshop on Distributed Computing (IWDC), in: LNCS, vol. 2918, 2003, pp. 205–214.

[20] S.S. Kulkarni, M. Arumugam, Transformations for write-all-with-collision model, in: Proceedings of the International Conference on Principles of Distributed Systems (OPODIS), in: LNCS, vol. 3144, Springer, 2003, pp. 184–197. [21] S.S. Kulkarni, M. Arumugam, SS-TDMA: A self-stabilizing MAC for sensor networks, in: S. Phoha, T.F. La Porta, C. Griffin (Eds.), Sensor Network Operations, Wiley-IEEE Press, 2006. [22] S.S. Kulkarni, M. Arumugam, Transformations for write-all-with-collision model, Comput. Commun. 29 (2) (2006) 183–199. (Elsevier). [23] A. Lachenmann, U. Muller, R. Sugar, L. Latour, R. Neugebauer, A. Gefflaut, Programming sensor networks with state-centric services, in: Proceedings of the 6th IEEE International Conference on Distributed Computing in Sensor Systems, DCOSS, 2010. [24] P. Levis, D. Culler, Maté: A tiny virtual machine for sensor networks, ACM SIGOPS Oper. Syst. Rev. 36 (5) (2002) 85–95. [25] P. Levis, N. Lee, M. Welsh, D. Culler, TOSSIM: Accurate and scalable simulation of entire TinyOS applications, in: Proceedings of the First International Conference on Embedded Networed Sensor Systems, SenSys, November 2003, pp. 126–137. [26] J. Liu, M. Chu, J. Liu, J. Reich, F. Zhao, State-centric programming for sensor–actuator network systems, Pervasive Comput. 2 (4) (2003) 50–62. [27] S. Madden, M. Franklin, J. Hellerstein, W. Hong, TinyDB: An acquisitional query processing system for sensor networks, ACM Trans. Database Syst. (TODS) (2005). [28] M. Mizuno, M. Nesterenko, A transformation of self-stabilizing serial model programs for asynchronous parallel computing environments, Inform. Process. Lett. 66 (6) (1998) 285–290. [29] L. Mottola, G.P. Picco, Programming wireless sensor networks: Fundamental concepts and state of the art, ACM Comput. Surv. (2011). [30] M. Nesterenko, A. Arora, Stabilization-preserving atomicity refinement, J. Parallel Distrib. Comput. 62 (5) (2002) 766–791. [31] R. Newton, M. Welsh, Region streams: Functional macroprogramming for sensor networks, in: Proceedings of the First Workshop on Data Management for Sensor Networks, DMSN, August 2004. [32] S. Sen, R. Cardell-Oliver, A rule-based language for programming wireless sensor actuator networks using frequence and communication, in: Proceedings of the Third Workshop on Embedded Networked Sensors, EmNets, May 2006. [33] A. Tavakoli, D. Chu, J. Hellerstein, P. Levis, S. Shenker, A declarative sensornet architecture, in: Proceedings of the International Workshop on Wireless Sensor Network Architecture, April 2007. [34] K. Terfloth, G. Wittenburg, J. Schiller, Rule-oriented programming for wireless sensor networks, in: Proceedings of the International Conference on Distributed Computing in Sensor Networks, DCOSS, 2006. [35] M. Welsh, G. Mainland, Programming sensor networks using abstract regions, in: Proceedings of the First USENIX/ACM Symposium on Networked Systems Design and Implementation, NSDI, March 2004. [36] K. Whitehouse, F. Zhao, J. Liu, Semantic streams: A framework for declarative queries and automatic data interpretation. Technical Report MSR-TR-2005-45, Microsoft Research, 2005, April.

Reza Hajisheykhi is a Ph.D. student in Computer Science and Engineering Department at Michigan State University. He received his M.Sc. in Computer Engineering from Sharif University of Technology in 2009. Currently, he is working in Software Engineering and Network Systems Laboratory (SENS Lab) under the supervision of Dr. Sandeep Kulkarni.

Ling Zhu is a Ph.D. student in Computer Science and Engineering Department at Michigan State University. She received his M.Sc. in Computer Science from Michigan State University in 2011. Currently, she is working in Software Engineering and Network Systems Laboratory (SENS Lab) under the supervision of Dr. Sandeep Kulkarni.

Mahesh Arumugam is a software engineer at Citrix Systems Inc. from 2011. He received his Ph.D. degree under the supervision of Sandeep Kulkarni from Michigan State University in 2006. His interests lie in Network Architectures and Protocols, Wireless Mesh Networks, and Distributed Computing.

R. Hajisheykhi et al. / J. Parallel Distrib. Comput. 77 (2015) 41–57 Murat Demirbas is an associate professor at the Computer Science and Engineering department of SUNY Buffalo. He leads the UBiComp Lab (University at Buffalo Ubiquitous Computing lab). His research interests are in the broad area of Distributed and Networked Systems, Distributed algorithms, Fault-tolerant Computing, Self-stabilization, Ubiquitous Computing, Wireless Sensor Networks, Smartphones, and Crowd-sourcing.

57

Sandeep Kulkarni received Ph.D. degree from the Ohio State university in 1999. After that, he joined Michigan State University. Currently he is an associate professor at the Department of Computer Science and Engineering at Michigan State University. His interests lie in Operating Systems, Distributed Systems, and Fault Tolerance.