CG-E2S2: Consistency-guaranteed and energy-efficient sleep scheduling algorithm with data aggregation for IoT

Accepted Manuscript CG-E2S2: Consistency-guaranteed and energy-efficient sleep scheduling algorithm with data aggregation for IoT Haneul Ko, Jaewook Lee, Sangheon Pack

PII: DOI: Reference:

S0167-739X(17)30582-4 http://dx.doi.org/10.1016/j.future.2017.08.040 FUTURE 3635

To appear in:

Future Generation Computer Systems

Received date : 10 April 2017 Revised date : 16 June 2017 Accepted date : 21 August 2017 Please cite this article as: H. Ko, J. Lee, S. Pack, CG-E2S2: Consistency-guaranteed and energy-efficient sleep scheduling algorithm with data aggregation for IoT, Future Generation Computer Systems (2017), http://dx.doi.org/10.1016/j.future.2017.08.040 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

CG-E2S2: Consistency-Guaranteed and Energy Efficient Sleep Scheduling Algorithm with Data Aggregation for IoT✩,✩✩ Haneul Koa , Jaewook Leea , Sangheon Packa,∗ a

School of Electrical Engineering, Korea University, Seoul, Korea

Abstract In data acquisition (DAQ)-based services of Internet of things (IoT), IoT devices sense data and transmit data to the application server through IoT gateway (GW). Due to the energy limitation of IoT devices, it is an important issue to increase energy efficiency of IoT devices. Meanwhile, when data from a huge number of IoT devices is individually transmitted, the data traffic volume can be significant. To resolve these issues, IoT devices and IoT GW can use sleep mode and data aggregation, respectively. However, when the IoT devices are in sleep mode for a long time and/or data are aggregated in IoT GW for a long time without any transmissions, the data can be inconsistent. In this paper, we propose a consistency-guaranteed and energy efficient sleep scheduling algorithm (CG-E2S2) with data aggregation. In CG-E2S2, the optimal sleep duration of IoT devices and aggregation duration in IoT GW are jointly determined by means of Markov decision process (MDP) with the consideration of energy efficiency of IoT devices, data traffic in networks, and data consistency. Evaluation results demonstrate that CG-E2S2 with the optimal policy outperforms the comparison schemes in terms of energy efficiency, data traffic volume, and data consistency. Keywords: Internet of things (IoT), Markov decision process (MDP), sleep, energy, data aggregation, data consistency.

Preprint submitted to Elsevier

April 3, 2017

1. Introduction Cisco Internet business solutions group predicts an explosive growth in the number of Internet of things (IoT) devices in the next few years [2]. Specifically, there will be 50 billion IoT devices connected to the Internet by 2020. These IoT devices enable users to be connected to the Internet anywhere and anytime [3]. Moreover, by exploiting the pervasive presence of things (e.g., such as radio frequency identification (RFID) tags, sensors, actuators, mobile phones, and etc.) which are able to interact and cooperate with each other, a variety of advanced IoT services can be provided [4]. In data acquisition (DAQ)-based services of IoT (e.g., heating, ventilation, and air conditioning (HVAC) monitoring and control systems in smart home), IoT devices sense data and transmit data to the application server through IoT gateway (GW) [5]. Then, the collected data can be analyzed in the application server. In such IoT services, IoT devices can use sleep mode where the radio units are turned off to achieve better energy efficiency [6]. Meanwhile, IoT GW can aggregate data from IoT devices and transmit the aggregated data in a batch manner to reduce the data traffic volume [5]. However, if the IoT devices are in sleep mode for a long time and/or data are aggregated in IoT GW for a long time without any transmissions, the data can be inconsistent. In this paper, we propose a consistency-guaranteed and energy efficient sleep scheduling algorithm (CG-E2S2) with data aggregation. In CG-E2S2, IoT GW commands IoT devices to change their mode to sleep mode with specific sleep durations. Also, IoT GW aggregates data and transmits data only after data are aggregated during a specific duration. In so doing, IoT devices can ✩ A preliminary version of this paper will be presented at the International Workshop on 5G Enabling Technologies for the Internet of Things (GET-IoT) 2017 (Conjunction with European Wireless 2017), May 2017 [1]. ✩✩ This work was supported by National Research Foundation of Korea Grant funded by the Korean Government (NRF-2014R1A2A1A12066986). ∗ Corresponding author Email addresses: [email protected] (Haneul Ko), [email protected] (Jaewook Lee), [email protected] (Sangheon Pack)

2

save their energy consumption and the data traffic volume can be reduced while maintaining high data consistency. Note that the needed degree of data consistency is dependent on the type of data (i.e., the type of IoT device). For example, emergency data (e.g., fire detection, cardiac standstill, and etc.) should be immediately delivered to the application server for maintaining a high data consistency. Whereas, consistency for casual data (e.g., temperature, humanity, and etc.) can be maintained at the high level even though such data is not immediately delivered to the application server. Therefore, the optimal sleep duration and the aggregation duration are dependent on the type of IoT device. To optimize the performance of CG-E2S2, a Markov decision process (MDP) problem is formulated and the joint optimal policy on the sleep duration and aggregation duration is obtained by a value iteration algorithm. Evaluation results demonstrate that CG-E2S2 with the optimal policy outperforms the comparison schemes in terms of energy efficiency, data traffic volume, and data consistency. The key contribution of this paper can be summarized: 1) to the best of our knowledge, this is the first work to jointly determine sleep duration for each IoT device and the aggregation duration in IoT GW while considering data consistency by means of the MDP formulation where two types of actions (i.e., action for sleep duration of IoT device and action for deciding whether to aggregate or transmit data) are defined; and 2) we present and analyze the evaluation results under various environments to assess the performance of the proposed scheme, which can provide guidelines for designing energy efficient IoT environments. The remainder of this paper is organized as follows. Related works are summarized in Section 2. Then, the system model and the detailed operation of CG-E2S2 are described in Section 3, while its MDP model is developed in Section 4. Further, evaluation results are presented in Section 5, and followed by concluding remarks in Section 6.

3

2. Related works Since energy efficiency is a challenging issue in IoT, there are many works in the literature [6, 7, 8, 9, 5]. Taneja [5] proposed a framework where IoT devices and IoT GW are in sleep mode as long as possible by considering quality of service (QoS) requirements to increase energy efficiency of IoT devices as well as IoT GW. Kaur and Sood [6] proposed an energy efficient architecture for IoT which consists of three layers, i.e., sensing and control, information processing and presentation layers, to predict the sleep interval of IoT devices based upon their remaining battery level, their previous usage history, and quality of information required for a particular application. Zhou et al. [7] designed an energy efficient index tree which is constructed through merging neighbor subregions and utilized it for collecting, querying, and aggregating data from IoT devices located in multiple regions. Similarly, in [8], Tang et al. divided the region evenly into grid cells and then organized an energy efficiency hierarchical clustering index tree by arranging these grid cells in a hierarchical manner. Meanwhile, Oro et al. [9] introduced object group mobility concept which can reduce the energy consumption to conduct location update. However, in these works, there is no optimization method to jointly determine sleep duration for each IoT device and the number of data aggregated in IoT GW while considering data consistency.

3. Consistency-Guaranteed and Energy Efficient Sleep Scheduling Algorithm (CGE2S2) with Data Aggregation Figure 1 shows the system model in this paper. We consider DAQ-based services where IoT devices collect data (i.e., sense something such as temperature, ventilation, and etc.) and send data to IoT GW. Then, IoT GW sends these data to the application server to examine and analyze these data. Since IoT devices have a limited battery power, IoT devices can adopt sleep mode where the

4

#RRNKECVKQP UGTXGT

5VQTCIG

#PCN[\GT

0GVYQTMU &CVCUVCVG    +Q6 )9

#EVKXGUVCVG 5NGGRUVCVG

+Q6 FGXKEGU

Figure 1: System model.

radio units are turned off (i.e., the only sensing units are turned on in sleep mode)1 . Generally, the radio unit consumes much more energy than the sensing unit [11] and the sensing unit consumes little energy [12]. That is, negligible energy is consumed in sleep mode. Since the radio unit is turned off in sleep mode, an IoT device in sleep mode cannot send data to IoT GW right after collecting data. Instead of sending data to IoT GW, the IoT device aggregates data during sleep mode. After the mode of the IoT device is changed to active mode, the IoT device sends the aggregated data to IoT GW. In contrast, when an IoT device is in active mode, the IoT device collects data and sends it immediately to IoT GW. After receiving data from IoT devices, IoT GW aggregates data to reduce congestion of networks2 . Since the needed degree of data consistency is dependent on the type of data, different aggregation size is decided according to the type of data, and therefore IoT GW classifies data according to IoT device. For example, data state [2,3,0] in Figure 1 denotes that the numbers of data from the 1st, 2nd, and 3rd IoT devices are 2, 3 and 0. 1

Since the radio and the sensing unit of IoT devices are separate units, IoT devices can perform sense something even though IoT devices is in sleep mode. 2 IoT GW can exploit in-network aggregation techniques (e.g., delta encoding [13]), which involve different ways to route data in order to combine data coming from different sources but directing toward the same destination and then reduce the amount of data to be transmitted over networks [14, 15]. However, such aggregation techniques are beyond the scope of this paper.

5

These are transmitted to the application server in a batch manner. By using sleep mode in IoT devices, better energy efficiency can be achieved. Also, traffic congestion can be mitigated by means of data aggregation in IoT GW. However, when IoT devices are in sleep mode for a long time and/or IoT GW aggregates data for a long time, data consistency can be degraded. Therefore, we propose CG-E2S2 with data aggregation to achieve better energy efficiency and reduce data traffic while maintaining high data consistency. In CG-E2S2, IoT GW commands IoT devices to change their mode to sleep mode with a specific duration. Note that this duration should be decided by considering the type of data (i.e., the type of IoT device). For example, the sleep duration for IoT device sensing emergency data (e.g., fire detection) should be set to small value. Also, IoT GW aggregates data and then transmits data when the aggregation duration is larger than or equal to a specific time to balance reducing traffic congestion and maintaining high data consistency. This duration is also dependent on the type of data (i.e., the type of IoT device). To optimize the performance of CG-E2S2, we formulate an MDP problem which jointly determines sleep duration for each IoT device and the aggregation duration in IoT GW. Figure 2 shows an operational example of CG-E2S2. At the first time, all IoT devices (i.e., IoT device 1, 2, and 3) are in active mode. Therefore, all IoT devices send data right after sensing data (Steps 1, 2, 4, and 5). When receiving data, IoT GW aggregates data in its buffer. These are classified according to IoT devices. For example, the data state after Step 2 is [1,0,1] which denotes that there is one data from devices 1 and 3, respectively, in IoT GW buffer. Since we assume that data from IoT device 3 is emergency data, it is immediately delivered to the application server for maintaining a high data consistency (Step 3). Meanwhile, when the aggregation duration for data from IoT device 2 reaches a certain time, IoT GW sends data from IoT device 2 to the application server to satisfy data consistency (Step 6). When IoT GW commands IoT devices 1 and 2 to sleep

6

+Q6 FGXKEGU

+Q6 FGXKEGU

+Q6 FGXKEGU

+Q6 )9

#RRNKECVKQP UGTXGT

#EVKXG 5NGGR

#EVKXG

#EVKXG

#EVKXG

6TCPUOKVFCVC

&CVCUVCVG

  

5GPUKPI 6TCPUOKVFCVC 5GPUKPI

   6TCPUOKVFCVC HTQO+Q6 FGXKEG

   6TCPUOKVFCVC 5GPUKPI 6TCPUOKVFCVC 5GPUKPI

      6TCPUOKVFCVC HTQO+Q6 FGXKEG

%QOOCPFUNGGR

5NGGR

  

%JCPIGOQFG

5NGGR

5GPUGFCVC

7 7

5GPUKPI

5GPUKPI 5GPUKPI #EVKXG

%JCPIGOQFG 6TCPUOKVFCVC

   6TCPUOKVFCVC HTQO+Q6 FGXKEG

#EVKXG

%JCPIGOQFG 6TCPUOKVFCVC

     

Figure 2: Operation example.

7

with sleep durations T1 and T2 (Step 7)3 , respectively, IoT devices 1 and 2 change their mode to sleep mode (Step 8). Note that IoT devices can save their energy proportional to their sleep durations. In this example, since we assume that T2 is larger than T1 , IoT device 2 can save more energy than IoT device 1. After being in sleep mode, when IoT devices 1 and 2 sense data (Step 9), they aggregate data instead of sending data to IoT GW. When the sleep duration of IoT device 1 is over, IoT device 1 changes its mode to active mode (Step 10). Then, IoT device 1 transmits aggregated data to IoT GW (Step 11). After that, IoT GW sends aggregated data from IoT device 1 to the application server (Step 12). When the sleep duration of IoT device 2 expires, IoT device 2 changes its mode to active mode (Step 13) and sends data to IoT GW (Step 14). Due to long sleep duration of IoT device 2, T2 , data sensed in Step 8 cannot be transmitted to the application server for a long time, which represents that data consistency of IoT device 2 can be degraded.

4. MDP formulation Sleep mode can save the energy of IoT devices. However, when an excessive long sleep duration is applied, data cannot be delivered to the application server, and thus data consistency can be degraded. Therefore, sleep duration should be determined with an appropriate value to balance energy efficiency of IoT device and data consistency. Meanwhile, since the aggregation duration in IoT GW also influences to data consistency, this duration is also decided by an appropriate value. Therefore, sleep duration for each IoT device and the aggregation duration in IoT GW should be jointly determined with the consideration on data consistency. Note that, since the needed degree of data consistency is dependent on the type of data (i.e., the type of IoT device), sleep durations and aggregation durations should be differently decided according to the type of data (i.e., the 3

In this example, since IoT GW decides that data sensed by IoT device 3 is urgent, IoT GW does not command IoT device 3 to change its mode to sleep mode.

8

Table 1: Summary of notations.

Notation

Description

St

State at the decision epoch t

At

Action chosen at the decision epoch t

τ

Duration of each decision epoch

A

Action set

AI

Action vector set for sleep duration

AD

Action vector set for aggregating or transmitting data

S

State space

M

State for the mode of IoT devices

P

State for the previous sleep action

D

State for the number of data aggregated in IoT GW

E

State for the time elapsed from the last transmission

ω1

Weighted factor to balance the reward and cost functions

ω2

Weighted factor to balance the transmission and inconsistency cost functions

ω3

Weighted factor to balance two inconsistency cost functions

ue

Energy saved during unit time

Ls

Unit connection setup cost

Ld

Unit data delivery cost

type of IoT device). To this end, we formulate an MDP model4 with five elements: 1) decision epoch; 2) action; 3) state; 4) transition probability; and 5) reward and cost functions [17, 18]. Subsequently, we introduce the optimality equation and a value iteration algorithm to solve the equation. Important notations for the MDP model are summarized in Table 1. 4.1. Decision Epoch A sequence Te = {1, 2, 3, ...} represents the time epochs when successive decisions are made. Random variables St and At denote the state and the action chosen at the decision epoch t ∈ Te , respectively. τ represents the duration of each decision epoch. 4

The MDP model represents a mathematical framework to model decision-making in situations in which outcomes are partially random and partially under the control of the decision maker [16]. Therefore, the MDP model appears suitable for deciding sleep duration and the number of data to be aggregated.

9

4.2. Action The action vector set, A, can be described as

A = {A1 , A2 , ..., ANP,A }

(1)

where NP,A means the total number of possible combinations of actions. Ak represents the kth possible combination of actions. At each decision epoch, IoT GW commands IoT devices to sleep during a certain duration and decides whether to aggregate data in its buffer or transmit data to the application server (i.e., IoT GW conducts two types of actions). Therefore, the one of vectors in the action vector set, A, can be constructed by concatenating two vectors: 1) action vector for sleep duration, AI ; and 2) action vector for deciding whether to aggregate or transmit data, AD . That is, A = AI ⊕ AD , where ⊕ means the concatenation between two vectors. Based on the current state information, IoT GW commands IoT devices to sleep during a certain duration. Therefore, AI can be represented by [aI1 , aI2 , ..., aINI ], where aIi denotes sleep duration commanded to the ith IoT device and NI is the number of IoT devices. Note that IoT GW commands only IoT devices in active mode and aIi = −1 means that IoT GW does not command the ith IoT device (i.e., the ith IoT device remains active mode). Meanwhile, IoT GW classifies data according to IoT devices and decides whether to aggregate data in its buffer or transmit data to the application server. Accordingly, AD can be denoted D D D by [aD 1 , a2 , ..., aNI ], where ai represents whether to aggregate or transmit data from the ith IoT

device. That is, aD i = 0 means that IoT GW aggregates data from the ith IoT device. On the other hand, aD i = 1 represents that IoT GW transmits data from the ith IoT device to the application server. Note that the aggregation durations are decided by AD .

10

4.3. State We define the state space S as

S=M×P×D×E

(2)

where M means the vector set that describes IoT devices’ phase (i.e., active/sleep mode and remained sleep duration of IoT devices). Also, P represents the vector set that illustrates previously commanded sleep durations, and D denotes the vector set which represents the number of data aggregated in IoT GW. On the other hand, E describes the vector set for the time elapsed from the last transmission to the application server. Meanwhile, M is described by

M = {M1 , M2 , ..., MNP,M }

(3)

where NP,M is the total number of possible combinations of IoT device phases. Also, Mk denotes the kth element of M, which is represented by

Mk = [m1 , m2 , ..., mNI ]

(4)

where mi represents the phase of the ith IoT device. That is, mi = 0 means the situation right after the ith IoT device changes its mode to active mode. Also, mi = −1 means that the ith IoT device is in active mode. Meanwhile, mi > 0 represents that the ith IoT device is in sleep mode and the remained sleep duration is mi .

11

P is represented by

P = {P1 , P2 , ..., PNP,P }

(5)

where NP,P is the number of possible combinations of previously commanded sleep durations. In addition, Pk is the kth possible sleep duration combination. Therefore, Pk can be denoted by

Pk = [p1 , p2 , ..., pNI ]

(6)

where pi denotes the sleep duration which is commanded previously to the ith IoT device. On the other hand, D is denoted by

D = {D1 , D2 , ..., DNP,D }

(7)

where NP,D is the number of possible combinations of numbers of data aggregated in IoT GW. Dk denotes the kth element of D, which can be represented by

Dk = [d1 , d2 , ..., dNI ]

(8)

where di describes the number of aggregated data from the ith IoT device. On the other hand, E is denoted by

E = {E1 , E2 , ..., ENP,E }

(9)

where NP,E is the number of possible combinations of time elapsed from the last transmission and Ek denotes the kth possible combination of time elapsed from the last transmission, which can be 12

represented by

Ek = [e1 , e2 , ..., eNI ]

(10)

where ei describes the time elapsed from the last transmission for data from the ith IoT device. 4.4. Transition Probability For simple description, we can think that IoT GW chooses two action vectors (i.e., AI and AD ) at each decision epoch. Then, M and P are influenced by chosen action AI . Also, D and E are affected by chosen action AD . Meanwhile, when the mode of IoT device is changed to active mode, the aggregated data in IoT devices during sleep mode is transmitted to IoT GW at the same time. At this time, the number of data transmitted to IoT GW is depended on the sleep duration. That is, D is influence by M and P. To sum up, the transition probability from the current state, S = [Mk , Pk , Dk , Ek ], to the next state, S ′ = [Mk′ , Pk′ , Dk′ , Ek′ ], can be described by

P [S ′ |S, A] = P [Mk′ |Mk , AI ] × P [Pk′ |Pk , AI ] × P [Dk′ |Dk , Mk , Pk , AD ] × P [Ek′ |Ek , AD ].

(11)

The transition probability of M can be derived as follows. Since the modes of each IoT device are independent, P [Mk′ |Mk , AI ] =

Q i

P [mi′ |mi , aIi ]. On the other hand, when the ith IoT device

is in active mode (i.e., mi = −1), IoT GW can command the ith IoT device to sleep. Then, the remained sleep duration for the ith IoT device in the next state, mi′ , is changed to sleep duration commanded to the ith IoT device, aIi . Therefore, P [mi′ |mi = −1, aIi ] can be derived as

P [mi′ |mi = −1, aIi ] =

    1, if mi′ = aIi    0, otherwise.

13

(12)

Note that IoT GW does not command the ith IoT device (i.e., aIi = −1) and then the ith IoT device remains in active mode (i.e., mi′ = −1). On the other hand, when IoT device is in sleep mode, the remained sleep duration decreases one by one. Therefore, the corresponding probability can be defined by

P [mi′ |mi 6= −1, aIi ] =

Q i

    1, if mi′ = mi − 1    0, otherwise.

(13)

Since the actions for sleep duration ordered to each IoT device are independent, P [Pk′ |Pk , AI ] = P [pi′ |pi , aIi ]. When IoT GW commands the ith IoT device with sleep duration aIi , the com-

manded sleep duration is changed to chosen action, aIi . Therefore, we have

P [pi′ |pi , aIi ] =

    1, if pi′ = aIi

   0, otherwise.

(14)

IoT GW classifies data according to IoT device and transmits it independently. Therefore, P [Dk′ |Dk , Mk , Pk , AD ] can be calculated by

Q i

P [di′ |di , mi , pi , aD i ]. Meanwhile, if the chosen action

for data from the ith IoT device, aD i , is 1 (i.e., IoT GW sends data to the application server) and the ith IoT device is not in active mode (i.e., mi 6= −1), di′ is always 0. Therefore, the corresponding probability is described as

P [di′ |di , mi 6= −1, pi , aD i = 1] =

    1, if di′ = 0

   0, otherwise.

(15)

When the ith IoT device is in active mode (i.e., mi = −1), it sends data immediately after sensing. Therefore, di increases by one at the next state with the probability that data is sensed by the ith IoT device within τ . We assume that the inter-data sensing time of the ith IoT device 14

follows an exponential distribution with mean 1/λdi . Then, the probability that data is sensed by the ith IoT device within τ is given by λdi τ [19]. On the other hand, when the chosen action for data from the ith IoT device (i.e., aD i ) is 1, data aggregated in IoT GW in advance is sent to the application server. Therefore, when IoT GW transmits data from the ith IoT device and the ith IoT device is in active mode (i.e., aD i = 1 and mi = −1), di′ is 1 or 0 according to the data sensing probability. The corresponding transition probability is derived as     λdi τ, if di′ = 1     P [di′ |di , mi = −1, pi , aD 1 − λdi τ, if di′ = 0 i = 1] =        0, otherwise.

(16)

On the other hand, when IoT GW aggregates data from the ith IoT device (i.e., aD i = 0), di′ is changed based on di . That is, when data is sensed by the ith IoT device within τ , di′ = di + 1. On the contrary, when data is not sensed by the ith IoT device within τ , di′ = di . Therefore, P [di′ |di , mi = −1, pi , aD i = 0] is given by     λdi τ, if di′ = di + 1     P [di′ |di , mi = −1, pi , aD 1 − λdi τ, if di′ = di i = 0] =        0, otherwise.

(17)

When the mode of the ith IoT device is changed from sleep mode to active mode (i.e., mi is changed from 0 to −1), the aggregated data in the ith IoT device is transmitted to IoT GW in a batch manner. That is, when mi = 0 and the number of aggregated data during (pi + 1) is k,5 di increases at the next state by k. Meanwhile, since we assume that the inter-data sensing time 5

The ith IoT device cannot transmit data during the sleep duration commanded previously to the ith IoT device (i.e., pi ) as well as the additional one decision epoch where mi is changed from 0 to −1.

15

of the ith IoT device follows an exponential distribution with mean 1/λdi , the average number of aggregated data during (pi +1) is (pi +1)λdi τ . Then, the probability that the number of aggregated data during (pi + 1) is k can be defined by the Poisson distribution with mean (pi + 1)λdi τ . Meanwhile, di′ is depended on whether IoT GW aggregates or transmits data from the ith IoT D D device (i.e., aD i ). That is, when ai = 1 and ai = 0, di′ = k and di′ = di + k, respectively, as

similar as (16) and (17). Therefore, the corresponding probabilities are defined by

P [di′ |di , mi = 0, pi , aD i = 1] =

and

P [di′ |di , mi = 0, pi , aD i = 0] =

    P ((pi + 1)λdi τ, k), if di′ = k    0,

    P ((pi + 1)λdi τ, k), if di′ = di + k    0,

(18)

otherwise

(19)

otherwise

where P (x, k) is the probability mass function (PMF) of the Poisson distribution with mean x. When the ith IoT device is in sleep mode (i.e., mi > 0), the ith IoT device cannot send any data to IoT GW. Therefore, P [di′ |di , mi > 0, pi , aD i ] can be described as

P [di′ |di , mi > 0, pi , aD i ]=

    1, if di′ = di

   0, otherwise.

(20)

Since IoT GW transmits data from each IoT device independently, the time elapsed from the last data transmission for each IoT device is also independent. Therefore, P [Ek′ |Ek , AD ] can be derived as

Q i

P [ei′ |ei , aD i ]. When IoT GW does not transmit data from the ith IoT device

(i.e., aD i = 0), the time elapsed from the last transmission increases. Meanwhile, when IoT GW transmits data from the ith IoT device (i.e., aD i = 1), the time elapsed from the last transmission 16

to the application server is 0. Consequently, the corresponding probabilities can be denoted by

P [ei′ |ei , aD i = 0] =

and

    1, if ei′ = ei + 1    0, otherwise

P [ei′ |ei , aD i = 1] =

4.5. Reward and Cost Functions

    1, if ei′ = 0

   0, otherwise.

(21)

(22)

To define the reward function, we consider energy saving by sleep mode. Meanwhile, we take into account the data traffic volume in networks and the data inconsistency to define cost function. First, the total reward function, r(S, A), is defined as

r(S, A) = ω1 f (S, A) − (1 − ω1 ) g(S, A)

(23)

where f (S, A) is the reward function related to energy saving whereas g(S, A) is the cost function with respect to the data traffic volume in networks and the data inconsistency. Also, ω1 is a weighted factor to balance f (S, A) and g(S, A). IoT GW commands only IoT devices in active mode (i.e., m = −1) to sleep with duration aI (> 0)6 . Then, energy can be saved proportional to sleep duration of each IoT device (i.e., aIi (> 0)). Therefore, f (S, A) can be defined by

f (S, A) =

6 I

 P    ue aIi , if mi = −1, aIi > 0 i

   0,

otherwise

a = −1 means that IoT GW commands the IoT device to remain active mode

17

(24)

where ue is the energy saved during unit time. The cost function related to the data traffic volume in networks and the data inconsistency, g(S, A), can be expressed by

g(S, A) = ω2 gT (S, A) + (1 − ω2 ) gC (S, A)

(25)

where gT (S, A) and gC (S, A) are the data traffic volume and inconsistency cost functions, respectively. In addition, ω2 is a weighted factor to balance gT (S, A) and gC (S, A). Let Ls and Ld be the unit traffic volume for the connection setup and the unit traffic volume for one data delivery from IoT GW to the application server, respectively. Then, gT (S, A) can be expressed by [21, 22]

gT (S, A) =

 P    (Ls + di Ld ), if aD i =1 i

   0,

(26)

otherwise.

Data inconsistency occurs due to following two reasons: 1) sleep mode of IoT device; and 2) data aggregation in IoT GW. Therefore, gC (S, A) can be represented by

S D gC (S, A) = ω3 gC (S, A) + (1 − ω3 ) gC (S, A)

(27)

S (S, A) and g D (S, A) are the inconsistency cost functions due to sleep mode of IoT device where gC C

and data aggregation in IoT GW, respectively. Also, ω3 is a weighted factor to balance two inconsistency cost functions. Since the inconsistency cost is proportional to the amount of aggregated data and the elapsed time from the aggregation [20], the inconsistency cost functions can be defined as follows. When IoT device is in sleep mode during aIi (i.e., the chosen action is aIi (> 0)), the average number of 18

data occurring during aIi can be calculated by aIi λdi τ . On the other hand, if data occurs right after the ith IoT device changes its mode to sleep mode, aIi can be considered as the maximum elapsed S (S, A) is given by time from the first aggregation. Consequently, gC

S gC (S, A) =

 P P A I    βi ai λdi τ + βiE aIi , if aIi > 0 i

i

   0,

(28)

otherwise

where βiA and βiE are coefficients reflecting the consistency sensitivity of data from the ith IoT device on the amount of data and the elapsed time from the first aggregation, respectively. As either βiA or βiE increases (or decreases), data from the ith IoT device becomes more (or less) sensitive to the consistency. When IoT GW chooses the aggregation action for the ith IoT device (i.e., aD i = 0), the amount A (S, A) can be of data and the elapsed time from the first aggregation are di and ei . Therefore, gC

represented as

A gC (S, A) =

4.6. Optimality Equation

 P P A    βi di + βiE ei , if aD i =0 i

i

   0,

(29)

otherwise.

To maximize the expected total reward and obtain the optimal policy, we choose the expected total discount reward optimality criterion [23, 24]. Let v(S) be the maximum expected total reward when the initial state is S. Then, we can describe v(S) as

v(S) = max v π (S) π∈Π

where v π (S) is the expected total reward when the policy π with an initial state S is given. 19

(30)

The optimality equation is given by [17] (

v(S) = max r(S, A) + A∈A

X

S ′ ∈S

′

′

λP [S |S, A]v(S )

)

(31)

where λ is a discount factor in the MDP model. λ closer to 1 gives more weights to the future rewards. The solution of the optimality equation corresponds to the maximum expected total reward and the optimal policy. To solve the optimality equation and to obtain the optimal policy, δ, we use a value iteration algorithm, as shown in Algorithm 1, where |v| = max[v(S)] for S ∈ S. Algorithm 1 Value iteration algorithm. Set v 0 (S) = 0 for each state S. Specify ǫ > 0, and set k = 0. k+1 2: For each state S,   compute v (S) by P ′ k ′ k+1 λP [S |S, A]v (S ) v (S) = max r(S, A) + 1:

A∈A

S ′ ∈S

If |v k+1 (S) − v k (S)| < ǫ(1 − λ)/2λ, go to step 4. Otherwise, increase k by 1 and return to step 2. 4: For each state s∈ S, compute the stationary optimal  policy P δ(S) = arg max r(S, A) + λP [S ′ |S, A]v k+1 (S ′ )

3:

A∈A

and stop.

S ′ ∈S

5. Evaluation Results For the performance evaluation, we compare the proposed scheme, SCG−E2S2 , with the following four schemes: 1) SN O−SLEEP where IoT devices do not use sleep mode and IoT GW transmits data to the application server by means of MDP; 2) SA−SLEEP where sleep durations of IoT devices are proportional to ω1 and inversely proportional to λdi (i.e., sleep durations of IoT devices are set j k to ξ λωd1 where ξ is a scaling factor)7 while IoT GW transmits data to the application server by i

7

Since ω1 is a weighted factor between reward and cost functions, a large value of ω1 means a situation where energy saving of IoT devices is more important than data consistency. Also, large λdi means that the probability that data to be transmitted occurs frequently. Therefore, in SA−SLEEP , sleep durations of IoT devices are proportional to ω1 and inversely proportional to λdi .

20

Table 2: The default values of λd , β A , and β E for each IoT device.

λd βA βE

IoT device 1

IoT device 2

IoT device 3

0.8 0.7 0.6

0.6 0.4 0.3

0.4 0 0

means of MDP; 3) SN O−AGG where IoT GW does not aggregate data while sleep durations of IoT devices are decided by MDP; and 4) SK−AGG where IoT GW aggregates data when the number of data is K and then transmits data in a batch manner while sleep durations of IoT devices are decided by MDP. In SA−SLEEP and SK−AGG , ξ is set to 3 and K is set to 5. The number of IoT devices is set to 3. The maximum number of data which can be aggregated in IoT GW is set to 5. The default rates of inter data sensing time, λd , and coefficients to reflect the consistency sensitivity of data from the ith IoT device on the amount of data and the elapsed time from the first aggregation, βiA and βiE , are summarized in Table 2. By measurement values in 3G networks [21], the average delay for connection setup is 444.6 ms and the average delay to deliver one data is 747.05 ms. The connection setup cost and the data delivery cost in this work consider the latency for the connection setup and the data delivery, respectively. Therefore, we can assume that Ls and Ld are proportional to the corresponding delay (i.e., 444.6 ms and 747.05 ms). On the other hand, Ld is normalized as 1. Then, Ls can be set to 0.59. Meanwhile, all weighted factors (i.e., ω1 , ω2 , and ω3 ) are set to 0.5. The time slot length τ is set to 1, whereas the energy saved during unit time, ue , is set to 1. For value iteration algorithm, λ and ǫ are set to 0.98 and 0.001, respectively. 5.1. Effect of IoT device type Figure 3 shows the expected total reward of each IoT device. In this result, ω3 is set to 0. That is, the inconsistency due to sleep mode of IoT device is not considered in this result. It can be seen that the expected total reward of SCG−E2S2 is the highest among all schemes regardless of IoT 21

20 15

Expected total reward

10 5 0 -5 S QA-E2S2

-10

S A-SLEEP -15

S NO-SLEEP S NO-AGG

-20

S K-AGG

-25 1

2

3

IoT device #

Figure 3: Effect of IoT device.

device, which means that SCG−E2S2 chooses the optimal action according to IoT device type by considering the consistency sensitivity of data (i.e., βiA and βiE ) and the inter-data sensing time. For example, since the data consistency sensitivity of IoT device 3 is low (i.e., β3A and β3E are 0), SCG−E2S2 commands to IoT device 3 to sleep with long sleep duration and aggregates data up to limit. Therefore, the expected total rewards of SCG−E2S2 and SK−AGG are the almost same. Note that the maximum number of data which can be aggregated in IoT GW and K of SK−AGG are set to the same number in our evaluation setting. 5.2. Effect of ue Figure 4 shows the expected total reward as a function of the energy saved during unit time, ue . It can be seen that the expected total rewards of all schemes except SN O−SLEEP increases as ue increases. This can be explained as follows. Large ue means that IoT devices can save more energy during unit time. Also, since SN O−SLEEP does not command IoT devices to sleep, it cannot receive any reward related to ue . Meanwhile, it can be found that the expected total reward of SCG−E2S2 is the highest among all comparison schemes. This can be explained as follows. In SCG−E2S2 , IoT GW commands IoT devices to change their mode with appropriate sleep durations by considering 22

100

S QA-E2S2 80

S A-SLEEP

Expected total reward

S NO-SLEEP 60

S NO-AGG S K-AGG

40

20

0

-20

-40 0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

ue

Figure 4: Effect of ue .

energy efficiency and data consistency of each IoT device. Also, to reduce data traffic, SCG−E2S2 aggregates data and then transmits data in a batch manner only after data are aggregated during a specific duration which can provide a sufficient degree of data consistency. That is, two tradeoffs 1) between energy efficiency and data consistency and 2) between data traffic volume and data consistency are jointly considered and optimized in SCG−E2S2 . On the other hand, it can be shown that when ue has a small value (i.e., ue = 0.2), the expected total rewards of SN O−AGG and SK−AGG are smaller than that of SA−SLEEP . This can be explained as follows. In SN O−AGG and SK−AGG , sleep durations of IoT devices are decided by MDP (i.e., the optimal sleep durations in that state/action space). However, it does not significantly influence the expected total reward due to small ue . Meanwhile, since SA−SLEEP transmits data to the application server by means of MDP, more reward can be achieved even though sleep durations of IoT devices are not optimally decided (i.e., sleep durations of IoT devices in SA−SLEEP are set j k to ξ λωd1 ). Also, it can be found that when ue has a large value (ue ≥ 0.4), the expected total i

rewards of SN O−AGG and SK−AGG is larger than the of SA−SLEEP . This is because the expected

total reward is more affected by the sleep durations than the transmission actions when ue is large. 23

100

1 0.9

Expected sleep duration ratio

Expected total reward

50

0

-50

S QA-E2S2 S A-SLEEP

-100

S NO-SLEEP S NO-AGG

-150

0.8 0.7 0.6 0.5 0.4

S QA-E2S2

0.3

S A-SLEEP S NO-SLEEP

0.2

S NO-AGG

0.1

S K-AGG -200

S K-AGG

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0.1

ω1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

ω1

(a) Expected total reward

(b) Expected sleep duration ratio Figure 5: Effect of ω1 .

5.3. Effect of ω1 Figure 5 shows the effect of the weighted factor, ω1 , to balance the reward and cost functions. In these results, ω2 is set to 0 to show the obvious tradeoff between energy efficiency and data consistency. From Figure 5(a) and (b), it can be seen that SCG−E2S2 operates adaptively even when the weighted factor ω1 is changed (i.e., the expected total reward of SCG−E2S2 is the highest among all schemes regardless of ω1 ). When ω1 is small, the data consistency is more important than the energy saving of IoT devices. Therefore, SCG−E2S2 commands IoT devices to sleep with short sleep duration. In particular, when ω1 is 0, it does not have any effect on the expected total reward that IoT devices change their modes to sleep modes. Therefore, SCG−E2S2 does not command IoT devices to change their modes to sleep modes in all states (i.e., IoT devices do not change their modes to sleep mode). As a result, in Figure 5(b), it can be found that the expected sleep duration ratio of SCG−E2S2 is 0.8 Meanwhile, as ω1 increases, the energy saving of IoT devices is more influential than the data consistency. Therefore, SCG−E2S2 commands IoT devices to sleep with longer sleep durations. Specifically, when ω1 ≥ 0.4, the expected sleep duration ratio is almost 8

The expected sleep duration ratio is defined as τs /τf where τs and τf is the expected sleep duration and the expected total simulation time, respectively.

24

300

40

S QA-E2S2 S A-SLEEP

250

Expected traffic volume

Expected total reward

20

0

-20

-40

S QA-E2S2

-60

S A-SLEEP S NO-SLEEP

-80

S NO-SLEEP S K-AGG

200

150

100

50

S K-AGG -100

0 1

2

3

4

5

6

7

8

9

10

1

Ls

2

3

4

5

6

7

8

9

10

Ls

(a) Expected total reward

(b) Expected traffic volume Figure 6: Effect of Ls .

one (i.e., IoT devices sleep during most simulation time). 5.4. Effect of Ls The effect of the unit connection setup cost, Ls , is demonstrated in Figure 6. Note that, since SN O−AGG transmits data without any aggregation, the performance of SN O−AGG is inferior to those of the other schemes. That is, the performance of SN O−AGG significantly decreases as Ls increases. For clear comparison among other schemes, SN O−AGG is excluded in these results. As shown in Figure 6(a), the expected total rewards of all schemes decreases with the increases of Ls . This is because the cost when IoT GW transmits the aggregated data to IoT platform increases as Ls increases. However, it can be found that the expected total reward of SCG−E2S2 is still larger than those of other comparison schemes, even though Ls has a large value. This can be explained by Figure 6(b). When Ls has a large value, IoT GW in SCG−E2S2 does not transmit data to reduce the connection setup costs. Therefore, the expected traffic volume of SCG−E2S2 does not significantly increase. Specifically, when Ls ≥ 8, the expected traffic volume of SCG−E2S2 rarely increases. On the other hand, other comparison schemes do not consider the connection setup cost and transmit data by following the fixed actions. Therefore, the expected traffic volume of other 25

40

Expected total reward

20

0

-20

S QA-E2S2 S A-SLEEP -40

S NO-SLEEP S NO-AGG S K-AGG

-60 0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

λd

Figure 7: Effect of λd .

comparison schemes continuously increases as Ls increases. 5.5. Effect of λd Figure 7 shows the effect of λd . λd means that the frequency of data occurrence. Therefore, when λd is large, shorter sleep duration should be assigned to IoT devices to maintain the data consistency. However, since SA−SLEEP cannot command appropriate sleep durations to IoT devices, this scheme cannot efficiently maintain the data consistency. Therefore, the expected total rewards of SA−SLEEP significantly decreases as λd increases. On the other hand, since SCG−E2S2 , SN O−AGG , and SK−AGG command IoT devices to sleep with the sleep duration derived by MDP (i.e., shorter sleep durations are commanded when λd is large), the consistency on data can be maintained and therefore the expected total rewards of these scheme slightly decrease. Note that the energy saving is proportional to the commanded sleep duration. Therefore, when the short sleep duration are commanded, less rewards are given. Meanwhile, since IoT devices in SN O−SLEEP do not use sleep mode, SN O−SLEEP cannot receive any rewards. In this situation, as λd increases, more frequent transmission is needed to maintain the data consistency, which reduce the expected total reward of SN O−SLEEP . 26

6. Conclusion In this paper, we propose a consistency-guaranteed and energy efficient sleep scheduling algorithm (CG-E2S2) with data aggregation. In CG-E2S2, the optimal sleep duration and aggregation duration are jointly determined by means of Markov decision process (MDP) with the consideration of energy efficiency of IoT devices, data traffic in networks, and data consistency. Evaluation results demonstrate that CG-E2S2 with the optimal policy outperforms the comparison schemes in terms of energy efficiency, data traffic volume, and data consistency. In future works, we will investigate how to implement and deploy the proposed algorithm in heterogeneous network environments.

References [1] H. Ko, J. Lee, and S. Pack, “Consistency-Guaranteed and Energy Efficient Sleep Scheduling Algorithm with Data Aggregation for IoT,” in Proc. International Workshop on 5G Enabling Technologies for the Internet of Things (GET-IoT) 2017 (Conjunction with EW 2017), May. 2017. [2] Cisco, “The Internet of Things - How the Next Evolution of the Internet Is Changing Everything,” [Online]. Available: http://www.cisco.com/c/dam/en_us/about/ac79/docs/innov/IoT_IBSG_0411FINAL.pdf [3] V. Gazis, “A Survey of Standards for Machine-to-Machine and the Internet of Things,” IEEE Communication Survey & Tutorials, vol. 19, no. 1, pp. 482-511, 1st quarter, 2017. [4] L. Atzori, A. Iera, and G. Morabito, “The Internet of Things: A survey,” Computer Networks, vol. 54, no. 15, pp. 2787-2805, Oct. 2010. [5] M. Taneja, “A Framework for Power Saving in IoT Networks,” in Proc. International Conference on Advances in Computing, Communications and Informatics (ICACCI) 2014, Sep. 2014. [6] N. Kaur and S. K. Sood, “An Energy-Efficient Architecture for the Internet of Things (IoT),” IEEE System Journal, to appear. [7] Z. Zhou, J. Tang, L. Zhang, K. Ning, and Q.Wang, “EGF-Tree: An Energyefficient Index Tree for Facilitating Multi-region Query Aggregation in the Internet of Things,” Personal and Ubiquitous Computing, vol. 18, no. 4, pp. 951-966, Apr. 2014. [8] J. Tang, Z. Zhou, J. Niu, and Q. Wang, “An Energy Efficient Hierarchical Clustering Index Tree for Facilitating

27

Time-correlated Region Queries in the Internet of Things,” Journal of Network and Computer Applications, vol. 40, pp. 1-11, Apr. 2014. [9] S. D’Oro, L. Galluccio, G. Morabito, and S. Palazzo, “Exploiting Object Group Localization in the Internet of Things: A Performance Analysis,” IEEE Transactions on Vehicular Technology, vol. 64, no. 8, pp. 3645-3656, Aug. 2015. [10] Low-Rate Wireless Personal Area Networks (LR-WPANs), Amendment 1: MAC Sublayer, IEEE 802.15.4e-2012. [11] H. Zhang, S. Olariu, J. Cao, and D. Johnson, Mobile Ad-Hoc and Sensor Networks Lecture Notes in Computer Science, MSN, 2007. [12] M. N. Halgamuge, M. Zukerman, and K. Ramamohanarao, “An Estimation of Sensor Energy Consumption,” in Proc. Electromagnetics Research B, 2009. [13] N. Samteladze and K. Christensen, “DELTA: Delta encoding for less traffic for apps,” in Proc. IEEE Local Computer Networks (LCN), Oct. 2012. [14] E. Fasolo, M. Rossi, J. Widmer, and M. Zorzi, “In-Network Aggregation Techniques for Wireless Sensor Network: A Survey,” IEEE Wireless Communications, vol. 14, no. 2, pp. 70-87, Apr. 2007. [15] P. Jesus, C. Baquero, and P. Almeida, “A Survey of Distributed Data Aggregation Algorithms,” IEEE Communication Survey & Tutorials, vol. 17, no. 1, pp. 381-404, 1st quarter, 2015. [16] H. Ko, G. Lee, D. Suh, S. Pack, and X. Shen, “An Optimized and Distributed Data Packet Forwarding in LTE/LTE-A Networks,” IEEE Transactions on Vehicular Technology (TVT), vol. 65, no. 5, pp. 3462-3473, May. 2016. [17] M. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. Hoboken, NJ: Wiley, 1994. [18] E.A. Feinberg and A. Shwartz, Handbook of Markov decision processes: Methods and applications. Kluwer Academic Publishers, 2002. [19] T. Guo, N. Wang, and R. Tafazolli, “Local Mobility Management for Networked Femtocells Based on X2 Traffic forwarding,” IEEE Transactions on Vehicular Technology (TVT), vol. 62, no. 1, pp. 326-340, Jan. 2013. [20] G. Lee, H. Ko, and S. Pack, “An Efficient Delta Synchronization Algorithm for Mobile Cloud Storage Applications,” IEEE Transactions on Service Computing (TSC), to appear. [21] C. Chuang, Y. Lin, and Y. Yeh, “Performance of Linear Mobile Data Transmission,” IEEE Transactions on Wireless Communications (TWC), vol. 10, no. 8, pp. 2451-2455, Aug. 2011. [22] H. Ko, J. Lee, and S. Pack, “An Opportunistic Push Scheme for Online Social Networking Services in Hetero-

28

geneous Wireless Networks,” IEEE Transactions on Network and Service Management (TNSM), to appear. [23] J. Pan and W. Zhang, “An MDP-Based Handover Decision Algorithm in Hierarchical LTE Networks,” in Proc. IEEE Vehicular Technology Conference (VTC) 2012 fall, Sep. 2012. [24] H. Tabrizi, G. Farhadi, and J. Cioffi, “A Learning-based Network Selection Method in Heterogeneous Wireless Systems,” in Proc. IEEE Global Telecommunications Conference (Globecom) 2011, Dec. 2011. [25] E. Stevens, Y. Lin, and V. Wong, “An MDP-based Vertical Handoff Decision Algorithm for Heterogeneous Wireless Networks,” IEEE Transactions on Vehicular Technology (TVT), vol. 57, no. 2, pp. 1243-1254, Mar. 2008.

29

Haneul Ko received the B.S. and Ph.D. degrees from Korea University, Seoul, Korea, in 2011 and 2016, respectively, both in School of Electrical Engineering. He is currently Postdoctoral Researcher in the mobile network and communications laboratory, Korea University, Seoul, Korea. His research interests include 5G networks, mobility management, mobile cloud computing, SDN/NFV, and Future Internet.

Jaewook Lee received the B.S. degree from Korea University, Seoul, Korea, in 2014. He is currently an M.S. and Ph.D. integrated course student in School of Electrical Engineering, Korea University, Seoul, Korea. His research interests include 5G networks, mobility management, and SDN/NFV.

Sangheon Pack received the B.S. and Ph.D. degrees from Seoul National University, Seoul, Korea, in 2000 and 2005, respectively, both in computer engineering. In 2007, he joined the faculty of Korea University, Seoul, Korea. From 2005 to 2006, he was a Postdoctoral Fellow with the Broadband Communications Research Group, University of Waterloo, Waterloo, ON, Canada. He was the recipient of KICS (Korean Institute of Communications and Information Sciences) Haedong Young Scholar Award 2013, IEEE ComSoc APB Outstanding Young Researcher Award in 2009, LG Yonam Foundation Overseas Research Professor Program in 2012, and Student Travel Grant Award at the IFIP Personal Wireless Conference (PWC) 2003. From 2002 to 2005, he was a recipient of the Korea Foundation for Advanced Studies Computer Science and Information Technology Scholarship. He was a publication co-chair of IEEE INFOCOM 2014, a co-chair of IEEE VTC 2010-Fall transportation track, a co-chair of IEEE WCSP 2013 wireless networking symposium, a TPC vice-chair of ICOIN 2013, and a publicity co-chair of IEEE SECON 2012. He is an editor of Journal of Communications Networks (JCN) and a senior member of the IEEE. His research interests include Future Internet, SDN/ICN/DTN, mobility management, mobile cloud networking, multimedia networking, and vehicular networks.

► Hane eul Ko

► Jaew wook Lee

► Sangheon Pack

Highlights

► Sleep and aggregation durations are jointly optimized and determined by Markov decision process ► Tradeoff between energy efficiency, traffic volume, and data consistency is investigated and optimized ► Valuable guidelines for designing energy efficient IoT environments are provided