Open-Loop cache update strategy by peeking into the future

Computer Communications 107 (2017) 49–59

Contents lists available at ScienceDirect

Computer Communications journal homepage: www.elsevier.com/locate/comcom

A Closed/Open-Loop cache update strategy by peeking into the future Lorenzo Maggi∗, Jérémie Leguay Mathematical and Algorithmic Sciences Lab, France Research Center, Huawei Technologies France SASU, France

a r t i c l e

i n f o

Article history: Received 21 September 2015 Revised 22 November 2016 Accepted 25 March 2017 Available online 2 April 2017 Keywords: Pre-fetching Popularity prediction Dynamic programming Model Predictive Control

a b s t r a c t We consider a scenario where predictions of future content popularity are available to the cache manager. Our aim is to demonstrate how to optimally exploit the content popularity predictions to derive the cache update strategy that minimizes the traffic at the core network. We formulate our cache update problem as a Closed/Open-Loop (COL) one, which at each time decision instant takes into account the current users’ content requests and the predicted future ones. In its original form, COL suffers from the curse of dimensionality. We then find a necessary and sufficient condition under which COL simplifies drastically and only requires a short prediction horizon and a limited computational complexity. Numerical investigations suggest that COL policy outperforms any cache update algorithm that only relies on past statistics. Remarkably, COL policy is robust with respect to popularity prediction errors. This advocates the exploitation of content popularity predictions for cache management purposes, even when predictions are largely inaccurate. © 2017 Elsevier B.V. All rights reserved.

1. Introduction According to recent studies [9], multimedia contents represent a significant portion of the whole Internet traffic (≈ 64%) and they will attain 80% by the year 2019. It is a common belief that, in order to withstand such traffic growth, a trade-off between bandwidth and memory has to be resolved. On the one hand, increasing the link bandwidth capacity allows the network to sustain heavier traffic without incurring congestion collapse. On the other hand, deploying caches across the network and close to the users allows to reduce the bandwidth utilization of network links altogether. In fact, each time a user requests a content that has been previously stored in the cache, the user can be directly served by the cache itself, thus saving precious core network resources. Due to the fact that storage costs are expected to decrease in the near future, future projections [19] promote the deployment of memory over an upgrade of link capacity. In such a memory/bandwidth trade-off, a fundamental role is played by the efficiency of the cache update strategy, which determines how and when the contents stored in a cache should be updated in order to minimize the traffic at the core network. Several cache update strategies have been studied over the years. Cache update strategies can be classified into replacement policies, that passively update the cache content at each new content request, and pre-fetching policies, that proactively retrieve contents are becoming popular (see [15] for a survey). Among re∗

Corresponding author. E-mail address: [email protected] (L. Maggi).

http://dx.doi.org/10.1016/j.comcom.2017.03.011 0140-3664/© 2017 Elsevier B.V. All rights reserved.

placement policies we find the classic recency-based (e.g., Least Recently Used – LRU), frequency-based (e.g., Least Frequently Used – LFU), function-based (e.g., GD-function) and randomized strategies (see [3] for a survey). Unlike replacement policies, pre-fetching generates extra-traffic on the core network. For this reason, a new successful paradigm first formalized by [12] has recently emerged, according to which the update of web caches should be performed during off-peak hours, so as to smooth out the traffic profile over time (see also [14]). The caching problem is then split in two phases, namely the placement phase, where the cache content is updated based solely on the statistics of user demands, and the delivery phase, where the cache caters for users requests but its content is left intact. Along this line of thought, in this paper we assume that the cache update instants are pre-determined and known a priori. As content popularity is the most crucial parameter in the cache update problem, all cache update policies tap into some sort of popularity estimation. Such estimation can be either implicit as for LRU (where the least recently requested content is estimated to be obsolete, hence it is evicted from the cache) or explicit as for LFU (where the frequency of requests for a content is an estimator of its popularity). There have also been several attempts to adopt exploration/exploitation techniques to jointly learn the popularity of contents and optimally update the cache (see, e.g., [2,5,13,23]). Nevertheless, it has already been noted in [25] that when user population is small such approaches suffer from a sharp degradation in terms of hit rate performance. For this reason, it has been recently proposed by [10] that, in hierarchical caching systems, the

50

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

caches at lower tiers gather statistics from higher-tier caches to speed-up popularity learning. To the best of our knowledge, all cache update policies (replacement and/or pre-fetching) proposed in the literature try to exploit the estimate of the current content popularity. On the other hand, a parallel and yet unexploited research stream studies techniques for predicting the future evolution of multimedia content popularity. There have been several studies showing that, by observing the early evolution of the number of views of video contents, it is possible to classify the future evolution of their popularity via different machine learning techniques with good accuracy (see, e.g., [1,8,11,16,20–22]). Motivated by the recent advances on content popularity prediction, our idea is then to exploit the available predictions to devise efficient cache update strategies. We then study a framework where popularity predictions represent an input to the cache update problem. We may consider that predictions are provided to the cache manager by higher-layer caches, being able to collect more accurate statistics on content requests and, eventually, learn the future popularity more efficiently. To see why predictions help in our scenario, let us consider the classic problem of refreshing the cache at a certain time instant with the aim of minimizing the traffic generated on the core network in the long-term. Typically, the process of retrieving a file can occur in two possible fashions, via a pull mechanism (i.e., the cache exploits users’ requests to update its content) or via a push mechanism (i.e., the cache proactively requests files that are not currently downloaded by any user). Consider first the extreme case where the traffic generated by the update process itself is negligible. Clearly, in this case it is optimal to greedily ensure that, at each decision instant tk , the cache stores the contents that are predicted to be the most popular ones during the following time interval [tk , tk+1 ]. On the other hand, if the cache update has a non-negligible traffic cost, then it is not trivial to figure out the structure of the optimal strategy. For instance, the greedy strategy may reveal itself to be wasteful when content popularity is very volatile, as the traffic generated by frantically updating the cache may surpass the core traffic that is saved during the delivery phase. Technically speaking, in this paper we drew inspiration from the Model Predictive Control (MPC) framework. MPC generally refers to a class of algorithms functioning on a discrete time grid and taking the instant decision on the basis of the predicted future evolution of the system. MPC algorithms adopt a very sensible paradigm: at each time step a long-run optimal strategy is calculated, as a function of the current (known) state of the system and of the currently available predictions of future state evolution. However, the computed strategy is implemented only in the current time step. In the following steps the algorithm receives as input updated predictions and the same procedure is repeated. Over the last decades, MPC has found several successful industrial applications, mainly in the field of chemicals, food processing, automotive and aerospace (see [17] for a survey), but it has never been applied to a caching scenario before.

tically simplified without loss of optimality, by only taking into account two step ahead popularity predictions. More specifically, in Section 2 we begin by describing the cache update scenario with popularity predictions. We then model in Section 2.1 the cache update problem as a COL one (Eqs. (3) and (4)). This choice is motivated by the fact that adopting a pure Closed-Loop (CL) approach (Eqs. (1) and (2)) would entail a daunting complexity. In fact, this would imply to solve at each cache update instant tk a dynamic programming formulation whose states encompass all the current and future possible configurations of cache content and users’ requests. Instead, our COL formulation considers i) the future content requests in expected value (and not the actual users’ future content requests, as in the pure CL approach) as well as ii) the instantaneous actual content requests (see Fig. 2). Technically speaking, COL calculates the future core network traffic i) via an Open-Loop (OL) sub-problem (see Eq. (4), which has lower complexity than CL and that we study in Section 3. Along the lines of the Model Predictive Control (MPC) classic framework, at each decision step the long-run COL problem is solved, but only the first step of the optimal strategy is implemented; in the next step, the popularity predictions are updated and the optimal solution is recomputed accordingly. Nevertheless, the solution to OL is in general still too complex as it suffers from the curse of state space dimensionality. For this reason, in Section 4.1, Theorem 1, we formulate a necessary and sufficient condition under which a straightforward Follow-theCrowd (FtC) strategy is optimal for the OL sub-problem. The optimality of FtC in the OL sub-problem presents a two-fold benefit. Firstly, the computational complexity of the solution to the original cache update COL problem simplifies drastically. Moreover, under FtC, the optimal cache update can be optimally calculated with very limited prediction capabilities. In fact, the popularity prediction over only the next two update periods is sufficient. Numerical simulations in Section 4.2 highlight the benefit brought by the integration of content popularity predictions into the cache update problem, even when those are largely inaccurate. We compare our COL optimal strategy with respect to the Store-the-Most-Popular (SMP) strategy, that stores in the cache the contents that have been the most popular in the last time period. The traffic gain is considerable (8% − 27%) when the popularity is sufficiently dynamic (i.e., the decay rate of its auto-correlation is between 0 and 0.5). Even more remarkably, the performance gain of our COL strategy gain is only slightly diminished (by at most 3%) when the prediction errors are large (up to 30%). These results advocate the exploitation of content popularity predictions for cache management purposes, even when predictions are largely inaccurate. Finally, we extend our open-loop OL formulation by taking into account the possibility to seamlessly update the cache, in continuous time, and not only at predefined discrete-time instants. We then show how to use a standard control theoretic framework to solve OL in continuous time.

1.1. Main contributions

2. System model

The main aim of this paper is to demonstrate how to optimally exploit the predictions on future content popularity to derive the cache update strategy that minimizes the traffic at the core network. We first formulate the cache update problem as a Closed/OpenLoop (COL) one. We do not constrain ourselves to updating the cache passively, via only pull traffic, but we also allow for proactive pre-fetching, via the push mechanism. We then demonstrate under which traffic regime our cache update solution can be dras-

We consider a communication scenario where N users download static multimedia contents from the network (see Fig. 1). Let M = {1, . . . , M} be the overall catalog of contents, all having the same size of V bytes. This assumption holds if all contents are divided into chunks of the same size, and the cache update policy operates on the chunks themselves. At each time t, the probability that a generic user is downloading content i is called pi (t). We also refer to pi (t) as the popularity of content i at time t. Thus, the probability that a user is idle at time t equals p0 (t ) =

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

51

Table 1 Table of notation symbols.

Fig. 1. System model.

 1 − i∈M pi (t ). We further consider that all parts of each content are equally popular. One cache with storage capacity C bytes is deployed in the system. For simplicity, and without much loss of generality, we will assume that C/V is an integer, i.e., the cache can store C/V contents in their entirety. We assume that if N(i) users request at the same time a multimedia content i, then the users download the content via N(i) uni-cast transmissions of rate ri (N) := r(N), being independent of the specific content i. If the cache also wants to retrieve the same content i, then the content can be directed to the cache first, that stores the content locally and then transmits it to one of the requesting users. Else, if the cache requests a content that no user is currently downloading, then the cache retrieves the content from the network via a dedicated uni-cast transmission of rate r(N). The rate r(N) generally decreases with the number of users N present in the system. For simplicity of notation, from now on we omit its dependence on N, and we define r := r(N). We now present the two main assumptions of this paper. Firstly, at each cache update instant tk we suppose that the popularity of the contents can be predicted over the future time window [tk , tk+T ], where T < ∞ is an integer characterizing the prediction time horizon. We call {  pi (t ), ∀ t ∈ [tk ; tk+T ]} the set of popularity predictions available to the cache controller at time tk . Secondly, the cache can only be updated at time instants t1 , t2 , . . . , which are fixed and determined a priori. Please find the list of symbols used throughout the paper in Table 1. 2.1. Problem statement The main goal of this paper is characterizing and computing the optimal update strategy for a single cache under the two assumptions presented above, i.e., 1) at time tk , the content popularity is known at all future instants t ∈ [tk ; tk+T ], and 2) the cache is updated at predetermined time decision instants tk . The performance of a cache update strategy is measured in terms of the generated traffic on the core network, that is to be minimized. Let us focus on the generic cache update instant tk . At time tk , the users’ content download requests materialize. We call Nk (i) the number of users requesting content i at time tk . Let Nk = ∪i Nk (i ) be the set of all content requests at time tk . Moreover, at time

tk pi (t)  pi (t ) C V r M N N(i) Nk S Ck CkCOL CkFtC +1

: : : : : : : : : : : : : :

JkOL ( Ck ) +1

:

ϕlCL (Cl , Cl−1 , Nl ) : xi (t)

φiPL (xi , t ) φiPS (xi , t )

: : :

cache update time instant popularity of content i at time t estimated popularity of content i at time t cache storage capacity content size download rate number of contents number of users number of users requesting content i at time t = ∪i Nk (i ), set of all content requests at time tk set of all possible contents stored in the cache set of contents stored in the cache at time tk optimal COL cache update strategy at time tk Follow-the-Crowd (FtC) open-loop strategy at future time instant tk+1 open-loop minimum expected core network traffic from instant tk+1 onward core network traffic generated during time interval [tl ; tl+1 ], knowing the current content requests Nl portion of content i stored in the cache at time t Pull traffic rate due to content i Push traffic rate due to content i

tk the predicted future content popularity over the time frame [tk ; tk+T ] is available. We are then interested in calculating the optimal cache update at the generic time instant tk given the information currently available, via an approach inspired by the Model Predictive Control (MPC) framework (e.g., [17]). This can be essentially accomplished in two ways, i.e., via a Closed-Loop (CL) or a Closed/Open-Loop (COL) problem solution. The former is optimal, but it is extremely demanding in terms of computational complexity. Let us first formulate the CL cache update at time tk . We first introduce ϕlCL (Cl , Cl−1 , Nl ) as the core network traffic generated during the generic time interval [tl ; tl+1 ] when the cache content is updated at time tl from Cl−1 to Cl . Let JlCL (Cl−1 , Nl ) be the minimum expected core network traffic from time instant l onward, assuming that Cl−1 is the cache content before the update at time tl . Then, the Closed-Loop (CL) optimal cache update CkCL can be formulated as (see also Fig. 2)

(CL)

CkCL = arg min Ck ∈S

ϕkCL (Ck , CkCL−1 , Nk ) + ENk+1 JkCL+1 (Ck , Nk+1 )

(1)

where the future core network traffic JkCL+1 (Ck , Nk+1 ) obeys the following closed-loop iterative Bellman’s equation:

(CL1) JlCL (Cl−1 , Nl ) = min Cl ∈S

⎧ CL CL ⎪ ⎨l ϕl (Cl , Cl−1 , Nl ) + ENl+1 Jl+1 (Cl , Nl+1 ), k+1≤l ≤k+T −2

⎪ ⎩ CL ϕl (Cl , Cl−1 , Nl ),

l = k + T − 1. (2)

The expected value in (1),(2) is evaluated with respect to the (predicted) future content requests Nk+1 and Nl+1 , respectively. More-

Fig. 2. Input/Output scheme of the main Closed/Open-Loop (COL) cache update problem at time tk . The optimal future OL traffic is calculated via an Open-Loop (OL) formulation (see Fig. 3). Notice that, in opposition to the OL formulation of the future traffic, the actual current requests Nk at time tk are input of the problem.

52

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

is given as input to the COL problem (see Fig. 2). OL Fig. 3. Input/Output scheme of the Open-Loop (OL) sub-problem at time tk . Notice that the produced output JkOL +1 considers the future expected number of requests E(Nk+1 ), . . . , E(Nk+T −1 ) (in the dashed box), while COL takes as input the actual requests Nk at time tk (cf. Fig 2).

over, the search space S is the set of possible contents stored in the cache

S=





C : C ⊂ M, |C | = C/V .

CkCOL = arg min Ck ∈S

OL ϕkCL (Ck , CkCOL −1 , Nk ) + Jk+1 (Ck ).

JlOL (Cl−1 ) = min

(OL)

We observe that both the instantaneous traffic cost ϕkCL and the future traffic JkCL+1 depend on the actual content requests N at time tk and tk+1 , respectively, i.e., Nk and Nk+1 . For this reason, the problem is called closed-loop. We also remark that, in the spirit of Bellman’s optimality principle, at each time instant l ≥ k + 1, CL (C , N Jl+1 l l+1 ) is computed under the assumption that the optimal CL cache update strategy will also be adopted in the future. Unfortunately, the state space of the CL cache update  problem M is formidably large for real systems. In fact, |S | = C/V , where C/V is the number of contents that can be stored by the cache. Moreover, the cardinality of all possible content requests Nl is in the order of MN . Therefore, state space dimensionality of CL, in (2),

M  the is in the order of C/V MN . For this reason, in this paper we turn to the study of a Closed/Open-Loop (COL) cache update optimization, with smaller state space. In Section 4.1 we will show how to further reduce the state cardinality and come up with a computable optimal COL solution. We now formally introduce our COL cache update problem, that we will solve in this work. The main difference with respect to CL lies in the fact that the optimal cache update is computed under the open-loop assumption that in the future, from time tk+1 onward, the cache controller will be oblivious to the actual content requests, and will only know them in expected value. More formally, the optimal Closed/Open-Loop (COL) cache update CkCOL at time step tk solves the following dynamic optimization problem:

(COL)

Section 3, and that can be formulated as (see also Fig. 3)

Cl ∈S

JkOL (C ) is the minimum expected core network traffic achievable +1 k in the future under the open-loop assumption, from instant tk+1 onward, when the initial cache allocation is Ck . Thus, COL computes the optimal cache update CkCOL by taking into account both the current content requests, described by the term ϕkCL (Ck , CkCOL , Nk ), −1 and the expected future requests during the time window [tk ; tk+T ] over which the content popularity can be predicted, equal to JkOL ( C ). +1 l We remark that, in contrast with the CL optimization problem, the open-loop future core network traffic JkOL (C ) does not depend +1 k on the actual content requests Nk+1 , which are only taken into account in expected value. JkOL (C ) is then the solution of an Open+1 k Loop (OL) dynamic programming problem that we will study in

k+1≤l ≤k+T −2

⎪ ⎩ OL ϕl (Cl , Cl−1 ),

(4)

l = k + T − 1.

ϕlOL (Cl , Cl−1 )

where is the expected future core network traffic during time interval [tl ; tl+1 ], and the expectation is evaluated with respect to the future content requests. In Sections 3 and 4 we will explicitly characterize the OL subproblem and main COL cache update problem, respectively. We will then show in Section 4.1 that under some conditions the complexity of the solution to OL can be drastically reduced, along with the associated COL problem. 3. Open-Loop (OL) minimum traffic We start our formal analysis by explicitly characterizing the OL sub-problem in (4), which is required in order to fully characterize the main COL problem in (3). The solution to OL can be formulated as a dynamic programming problem, under the open-loop assumption that the cache controller will oblivious to the actual requests of users in the future, and can only predict them in expected value. More specifically, we are interested in computing the minimum achievable future traffic JkOL (C ), over the future time window +1 k [tk+1 ; tk+T ], given that Ck is the cache content before the first update in the future at time tk+1 . We recall that the Open-Loop (OL) problem is defined as follows:

JlOL (Cl−1 ) = min

(OL)

Cl ∈S

(3)

We first notice that the COL problem in (3) is iterative in k, since it implicitly assumes that at time tk−1 the cache is updated via an analogous COL procedure, resulting in CkCOL . The term −1

⎧ OL OL ⎪ ⎨l ϕl (Cl , Cl−1 ) + Jl+1 (Cl ),

⎧ OL OL ⎪ ⎨l ϕl (Cl , Cl−1 ) + Jl+1 (Cl ),

k+1≤l ≤k+T −2

⎪ ⎩ OL ϕl (Cl , Cl−1 ),

l = k + T − 1.

where ϕl (Cl , Cl−1 ) is the expected traffic on the core network during time interval [tl ; tl+1 ), depending on the (random) content requests and under the (deterministic) cache update Cl−1 → Cl . In the following we will focus on the computation of the term ϕl (Cl , Cl−1 ). For this purpose, it is convenient to define Dlin ⊂ C l−1 and Dlout ⊂ Cl−1 as the set of contents that are retrieved by and erased from the cache at time tl , respectively. At time tl the cache is updated in the following way (see Fig. 4):





Cl = Cl−1 \ Dlout ∪ Dlin .

(5)

Since all contents have the same size and the cache is assumed to be always full, at each time tl there is an equal number of contents being retrieved by and erased from the cache. Thus, |Dlin | = |Dlout |. 3.1. Traffic rate We compute the total traffic ϕl (Cl , Cl−1 ) as the integral of the traffic rate over the interval [tl , tl+1 ]. Thus, in this section we aim at

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

53

M Cl−1

Dl Dlout

Dlin

Dl

Cl Dlin Fig. 6. A content is pushed to the cache at time tk when the cache wants to retrieve it from the core network but no user is currently requesting it.

Fig. 4. Cache update notation at time tl .

traffic rate Φi(xi, t) pull rate φPL i (xi , t)

traffic rate Φi(xi, t)

cache update rate (V x˙ i(t))+

push rate φPS i (xi , t) pull rate φPL i (xi , t)

Fig. 5. A content is pulled at time tk when it is requested by at least one user and it is not stored in the cache. The content can be directed to the cache first, that may store the content locally and then transmits it to the user.

Fig. 7. Push and pull rate for content i, depending on the required cache update rate (V x˙ i (t ))+ .

calculating the traffic rate at the generic time t ∈ [tl , tl+1 ]. We first observe that the total traffic on the core network has two different components: the pull traffic, which is initiated by the users and that occurs whenever a content user request cannot be fulfilled by the cache, and the push (or also “pre-fetching”) traffic, which is generated by the cache itself which proactively retrieves content from the core network.

that is being pulled at time t is greater than or equal to the cache update rate V x˙ i (t )+ dt , then the cache update for content i at time t can occur via only the pull mechanism. CL the other hand, if φiPL (xi (t ), t )dt < V x˙ i (t )+ dt , then the remaining increment V x˙ i (t )+ dt − φiPL (xi (t ), t )dt has still to be pushed to the cache directly from the core network. Consequently, the push rate φiPS (xi , t ) writes as follows:

Pull traffic rate. Whenever a portion of content i is requested by a user but is not stored at the cache, it needs to be retrieved from the core network (see Fig. 5). In this case, we say that a pull event has occurred, and φiPL (t ) is the core network traffic rate at time t associated to it. We now express the pull traffic rate associated to content i. Let us first define xi (t) ∈ [0; 1] as the portion1 of content i stored in the cache at time t. Hence, in our case2 xi (t ) = (t − tl )r/V at time t ∈ [tl ; tl + V/r], for a content i that is being stored in the cache at rate r, and xi (t ) = 1 − (t − tl )r/V for a contents i that is being erased at the same rate r. Since the inner content popularity is uniform, the probability that a user is downloading (at rate r) a packet of content i that is not present in the cache equals  pi (t )(1 − xi (t )). Moreover, we consider that there is a null probability that two or more users are requesting the same content packet at the same time instant. Therefore, φiPL (xi , t )/r equals the number of users that are pulling content i from the core

network at time t, which is a binomial variable with parameters N,  pi (t )(1 − xi (t )) . Its expected value equals the sum of pull rates over all users, i.e.:

(7)





E φiPL (xi , t ) = Nr  pi (t ) 1 − xi (t ) .

(6)

Push traffic rate. The push traffic component accounts for the expected traffic generated by pre-fetching. Formally speaking, assume that the evolution of the content allocation xi (t) is fixed. The amount of content i that is retrieved by the cache during the time infinitesimal interval [t ; t + dt ] equals the requested cache update rate V x˙ i (t )+ multiplied by the infinitesimal interval duration dt. While a content is being pulled from the core network by a user, it can also be retrieved by the cache via the same connection (see Fig. 6). If the (random) amount of content φiPL (xi , t )dt 1 For the sake of simplicity, we consider xi (t) to be a real variable. To be more precise, if the content is partitioned into n = V/z chunks of equal size z, then xi should take values in [0, n1i , n2i , . . . , nin−1 , 1]. i 2

Observe that V/r is the time required to retrieve a content in its entirety.



+ φiPS (xi , t ) = V x˙ i (t ) − φiPL (xi , t ) .

Total core network traffic rate. We are now in the position to compute the exact expression of the expected traffic rate φ i (xi , t), defined as the sum of pull and push traffic:



φi (xi , t ) = E φiPL (xi , t ) + φiPS (xi , t ) .

(8)

From expressions (7) and (8) we first observe that if V x˙ i (t ) ≥ φiPL (xi (t ), t ) then φi (xi , t ) = V x˙ i (t ), otherwise φi (xi , t ) = φiPL (xi (t ), t ); in other words (see also Fig. 7),



φi (xi , t ) = E max V x˙ i (t ), φiPL (xi (t ), t ) . Therefore, we can write



φi (xi , t ) = E V x˙ i (t )1I(φiPL (xi ,t )≤V x˙ i (t )) + φiPL (xi , t )1I(φiPL (xi ,t )>V x˙ i (t ))

 = V x˙ i (t ) Pr φiPL (xi , t ) ≤ V x˙ i (t ) + 

 + E φiPL (xi , t )φiPL (xi , t ) > V x˙ i (t ) . (9) We already noticed that φiPL (xi , t )/r is a binomial variable of parameters (N,  pi (t )(1 − xi (t ))). Then, the final expression of the traffic rate on the core network follows. Proposition 1. The expected traffic rate on the core network at time t ∈ [tl ; tl+1 ] associated to content i writes

  φi (xi , t ) = V x˙ i (t )Fi V r−1 x˙ i (t ) +

N 

jr fi ( j )

(10)

j= Vr x˙ i (t ) +1

where Fi and fi are the cumulative and probability distribution function of a binomial distribution of parameter (N,  pi (t )(1 − xi (t ))), rej spectively, i.e., Fi ( j ) = q=0 fi (q ) and

 

fi ( j ) =



j N  pij (t ) 1 − xi (t ) 1− pi (t )(1 − xi (t )) j

N− j

.

54

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

We remark that the expression of the core network traffic rate in (10) is valid in all possible cases, i.e., when the content i is being retrieved from the cache (x˙ i > 0), erased from the cache (x˙ i < 0), is not stored in the cache and not retrieved (xi = 0), or it is cached and not erased (xi = 1). 3.2. Total traffic



ciout (tl ) := ciout (tl ) =



tl +V/r

tl

Nr  pi (t ) dt

≈ NV  pi (tl ),

(16)

i ∈ Dlout .

in

We note that C l−1 ∩ Dl is the set of contents that are not cached at time tl− and that are not retrieved by the cache at time tl . Therefore, xi (t ) = x˙ i (t ) = 0 over the whole interval [tl ; tl+1 ] for in

After computing the core network traffic rate φ i (xi , t) at the generic time instant t ∈ [tl ; tl+1 ) as in (10), we are now ready to calculate the total traffic on the core network over the whole time interval [tl ; tl+1 ), that equals



ϕ (Cl , Cl−1 ) = OL l

tl+1

tl

i∈M

φi (xi , t ) dt.

⎧ for the incoming i ∈ Dlin ⎨lxi (t ) = (t − tl )r/V, xi (t ) = 1 − r (t − tl )/V, for the outgoing i ∈ Dlout ⎩ x˙ i (t ) = 0, else.

(11)

ϕlOL (Cl , Cl−1 )

We conveniently split the total traffic into four terms, whose meaning will be explained in the following, as



ϕ (Cl , Cl−1 ) =

ciin



(tl ) +

ciout



(tl ) +

i∈Dlout

i∈Dlin

in i∈C l−1 ∩Dl

ci (tl ) +



di (tl ).

i∈C l

(12) Let us start by computing ciin (tl ), being the traffic due to the retrieval of content i ∈ Dlin over the cache update interval [tl ; tl + V/r]. To this purpose, we specialize the expression of the core network traffic rate φ i (xi , t) in (10) to the case of an incoming content i ∈ Dlin by taking into account the expressions in (11), as

ciin (tl ) =



tl +V/r

tl

 =

tl +V/r

tl

 =

tl +V/r

tl

φi (xi , t ) dt rFi (1 ) + r





r 1− pi (t ) 1 −





V

(N + 1 )  pi (tl )

j f i ( j ) dt r (t − tl ) V

r tl

 = ≈

tl +V/r

tl

N 

(14)

k fi (k ) dt

k=1 tl +V/r

N  pi (tl )r 2 (t − tl ) dt V

NV  pi (tl ), 2

Nr  pi (t ) dt in

i ∈ C l−1 ∩ Dl .

(17)



tl+1

tl +V/r

Nr  pi (t )dt ,

i ∈ Cl ,

(18)

and di (tl ) = 0 for all contents i ∈ Cl stored in the cache. Finally, by putting all the expressions in (13) and (15) or (16),(17) and (18) together we can finally compute the total open-loop traffic ϕlOL (Cl , Cl−1 ) over the time interval [tl ; tl+1 ], as in expression (12). We point out that we implicitly assumed that the interval duration tl+1 − tl between two successive cache updates is higher than the time needed to retrieve a content, i.e., tl+1 − tl ≥ V/r for all l’s. 3.3. Optimal open-loop cache update strategy Having computed in (12) the exact expression of the core network traffic ϕlOL (Cl , Cl−1 ), the Open-Loop (OL) cache update problem in (4) is now finally fully characterized. The optimal OL cache update strategy CkOL , . . . , CkOL solves the classic backward induc+T −1 tion optimality relations (cf. with (4)):

ϕkOL+T −1 (Ck+T −1 , CkOL+T −2 ) OL OL ϕlOL (Cl , Cl−1 ) + Jl+1 ( Cl ),

∀ Cl−1 ∈ S, k + 1 ≤ l ≤ k + T − 2

where the approximation in (14) holds when  pi (t ) is constant during the time interval [tl ; tl+1 ]. We now focus on the generic outgoing content i ∈ Dlout that is being erased at rate V x˙ i (t ) = −r from the cache. If a content being erased from the cache can be  t +V/r still accessed by a user, then ciout (tl ) equals exactly t l φi (xi , t ). l Thus,



di (tl ) =

Cl ∈S

(13)

1 NV  pi (tl ), i ∈ Dlin 2

tl +V/r

Finally, for all contents i ∈ Dl in the cache that are not updated, the traffic ci (tl ) = 0 is null since they are entirely found in the cache. We now focus on the time interval [tl + V/r; tl+1 ], when no cache update occurs and the updated cache content is Cl . In this case, only the pull traffic component is present, and the expected core network traffic equals

ClOL = arg min





NV  pi (tl ),

Ck+T −1 ∈S

N

r (t − tl ) dt V

1 − (1 −  pi (tl ))N+1 +

ciout (tl ) := ciout (tl ) =

tl

CkOL +T −1 = arg min

j=2

+ rN  pi (t ) 1 −

≈

N 



ci (tl ) = ≈

By using the notation defined in Section 4 we can say that

OL l

all z ∈ C l−1 ∩ Dl . The associated core network traffic has only the pull component, and it simply writes

(15)

i ∈ Dlout .

Else, if the cache cannot serve a user requesting content i while content i is being erased, then the associated core network traffic ciout (tl ) simply writes

where by convention CkOL := CkCOL . Therefore, the optimal cache update strategy can be computed via a standard shortest path approach (see, e.g., [6]) on the trellis defined by the set of cache states S (see Fig. 8). In fact, at each time step tl , the set of possible actions in each state is exactly S: the cache can be hypothetically refilled with any subset of C/V contents. Each of these action has a different cost ϕ (l, Cl , Cl−1 ), depending on the previous state Cl−1 . The state at the next step clearly coincides with the action Cl taken at step tl . Unfortunately, as it is common for dynamic programming formulations, OL suffers from the well-known curse of dimensional M ity (first observed by [4]). The state cardinality |S | = C/V of the dynamic programming problem is in fact daunting for realistic values of the catalog size and cache size. For this reason, we show a necessary and sufficient condition under which a simple strategy called “Follow-The-Crowd” is optimal in the OL cache update sub-problem. 4. Closed/Open-Loop (COL) cache update After having characterized the Open-Loop (OL) sub-problem in the previous section, we can finally attack our main Closed/OpenLoop (COL) cache update problem, formulated in (3) and that we

ϕ OL

k+ 1 (C  k+ 1 ,C k)

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59



 Ck+1

 Ck+2

 Ck+T−1

 Ck+1

 Ck+2

 Ck+T−1

 C k) Ck+1

 Ck+2

 Ck+T−1

Ck+2

Ck+T−1

,C 1

)

k

C k+

L ( O 1 ϕ k+

,  C +1 OL 1( k + ϕk

OL ϕk+1 (Ck+1 , Ck )

Ck

55

Ck+1

Fig. 8. Dynamic programming trellis structure to solve the Open-Loop (OL) cache update problem with only 4 states. For instance, this can be obtained with a 4 content catalog (M = 4) and a cache with content capacity equal to 1 or 3 (|S | = 1 or |S | = 3). In the former case, S = {1, 2, 3, 4}, while in the latter S = {{2, 3, 4}, {1, 3, 4}, {1, 2, 4}, {1, 2, 3}}.

recall here below:

(COL)

CkCOL = arg min Ck ∈S

OL ϕkCL (Ck , CkCOL −1 , Nk ) + Jk+1 (Ck ).

In order to fully characterize COL, we still need to the compute the term ϕkCL (Ck , CkCOL , Nk ), accounting for the expected core network −1 traffic generated during the time interval [tk ; tk+1 ] when the cache update is CkCOL → Ck and the set of users’ content requests is Nk . −1 For the sake of analysis simplicity, we assume that at time tk all users request simultaneously the contents since their beginning and will download them entirely. Then, similarly to the OL sub-problem, we can split ϕkCL (Ck , CkCOL , Nk ) into four terms of the −1 kind:

ϕkCL (Ck , CkCOL −1 , Nk ) =



i∈Dkin

+

 ciin (tk ) +

 ciout (tk )

i∈Dkout

 COL



 ci (tk ) +



di (tk )

i∈C k

in

i∈C k−1 ∩Dk

whose meaning will be revealed in the following. The term  ciin (tk ) accounts for the traffic generated by the retrieval of content i from the core network. If there are Nk (i) ≥ 1 users requesting content i at time tk , then  ciin (tl ) = V Nk (i ) and the cache retrieves the content via one of the Nk (i) uni-cast connections. In this case, the traffic has only the pull component. Conversely, if there is no user requesting it, then a special connection is set up to serve the cache via a push mechanism, and  ciin (tk ) = V . Therefore,

 ciin (tk ) = V (Nk (i ) + 1I(Nk (i )=0) ),

i ∈ Dkin .

(19)

 ciout (tk )

The term stands for the traffic generated by the contents Dkout that are being erased from the cache. Analogously to the computation of ciout , it can be computed under two different assumptions. If we assume that a content being erased from the cache cannot be accessed by any user requesting it (even if the requested packet is still temporarily stored in the cache), then 

 ciout (tk ) :=  ciout (tk ) = V Nk (i ),

i ∈ Dkout .

(20)

Otherwise, we can assume that when a user is requesting a content i ∈ Dkout that is being deleted from the cache, there is probability 1/2 that the packet being requested is still found in the cache. Then, 

 ciout (tk ) :=  ciout (tk ) =

V N ( i ), 2 k

i ∈ Dkout .

(21)

The pull traffic  ci (tk ) associated to a content i that is not in the cache at time tk−1 and is not being retrieved by the cache at time tk , i.e., i ∈

COL C k−1

∩

 ci (tk ) = V Nk (i ),

in Dk ,

equals COL

in

i ∈ C k−1 ∩ Dk .

(22)

Clearly, the contents in CkCOL \ Dkout that are left untouched in the −1 cache generate no traffic on the core network. The three terms above account for the traffic generated during the cache update interval [tk ; tk + V/r], where V/r is the download (or retrieval) time for each content. Finally, the pull traffic di (tk ) over the remaining part of the interval [tk + V/r; tk+1 ] is defined as t in (18), i.e., di (tk ) = k+1 Nr  pi (t )dt for all ∈ C k . tk +V/r

In the next sections we can finally focus on the actual computation of the COL optimal cache update CkCOL . 4.1. A drastic computational complexity reduction We remarked in Section 3.3 that the dynamic programming calculation of the open-loop minimum traffic JOL , and consequently also of the COL cache update strategy, suffers from the common curse of dimensionality. Motivated by this, we now study how to reduce the complexity of the COL problem while preserving the optimality of its solution, by exploiting the special structure of the problem at hand. Specifically, in Section 4.1.1 we provide necessary and sufficient conditions under which an easily computable cache update strategy called “Follow-the-Crowd” is optimal in the OL sub-problem. We then finally show in Section 4.1.2 how this impact on the computation of the optimal COL cache update strategy. 4.1.1. Follow-the-Crowd We now aim at designing an easily computable strategy for OL which is optimal under certain conditions. Let us start by providing an intuition. If a content is popular at time tl , then it generates high pull traffic. Hence, with high probability at time tl there is at least one user requesting it, and the expected incremental push traffic to retrieve it from the network (if not already present in the cache) is small. Motivated by this simple intuition, we study a simple strategy dubbed “Follow-the-Crowd” (FtC), prescribing to persistently fill the cache with the currently most popular contents. We study under which scenario FtC is optimal in the open-loop sub-problem OL, and we next show in Section 4.1.2 how FtC greatly simplifies the calculation of our original COL cache update problem. More formally, the FtC strategy ensures that at each time decision step tl the most popular contents ClFtC are stored in the cache, where

ClFtC = {i1 (tl ), . . . , iC/V (tl )}  tl+1   s.t. pi1 (tl ) (t )dt ≥ · · · ≥ tl

tl

tl +1

 piM (tl ) (t )dt.

In an ideal scenario, where updating the cache at each time tl comes at no cost in terms of core network traffic (i.e., ciin = ciout =

56

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

ci = 0 for all i ∈ M), FtC is clearly optimal. By invoking a continuity property of the optimal strategy in the problem’s parameters, we show that the FtC strategy is still optimal for all retrieving costs ciin small enough. Theorem 1 (FtC optimality conditions). The FtC cache update policy is optimal for the OL sub-problem in (4), i.e., C OL = C FtC , if and only if, for all i ∈ ClFtC and j ∈ / ClFtC , out ciin (tl ) + cin j (tl+1 ) ≤ di (tl ) − d j (tl ) − c j (tl )+

− ciout (tl+1 ) + ci (tl ) + ci (tl+1 )

(23)

for all l = k + 1, . . . , k + T − 2 and, at the final step

ciin (tk+T −1 ) ≤ di (tk+T −1 ) − d j (tk+T −1 ) − cout j (tk+T −1 ) + ci (tk+T −1 ). (24)  Proof. We will prove this result by a classic exchange argument. Let us first consider k + 1 ≤ l ≤ k + T − 2. (⇒) Assume that C = Ck+1 , . . . , Ck+T −1 defines an optimal cache update strategy. Assume that C FtC = C. Then there exists a cache update instant tl , l < k + T − 1, at which content j ∈ Cl but j ∈ / ClFtC . Then there necessarily exists content i = j such that i ∈ ClFtC but i ∈ / Cl . We now define a slightly different cache update strategy C  which is equal to C except for Cl = Cl \ { j} ∪ {i}. We then evaluate the total cost





difference between the strategies C and C  . We denote JkOL C , C +1 k  the traffic when the cache update strategy C is employed. If l ≤

for all l = k + 1, . . . , k + T − 2 and, at the final step,



ciin (tk+T −1 ) ≤

tk+T −1





+ cout j

(tl )

+ ciout

(tl+1 ) − ci (tl ) − ci (tl+1 )

(25)

where the equality is attained when i ∈ Cl−1 ∪ Cl ∪ Cl+1 and j ∈ Cl−1 ∩ Cl ∩ Cl+1 . Then by inspection

we  conclude that condition  ≤ J OL (C , C ). Therefore if (23) is sufficient to ensure JkOL C , C k +1 k+1 k

(23) holds, then whenever a popular content belonging to C FtC is swapped with an unpopular content ∈ / C FtC then the total cost does not increase with respect to the optimal strategy, which proves that C FtC is also optimal. Therefore by induction we can claim that C FtC is optimal. (⇐) We now prove by contradiction that condition (23) is necessary for C FtC to be optimal. Consider a cache update strategy defined by C which equals C FtC except for Cl = ClFtC \ {i} ∪ { j}. Moreover, i ∈ Cl−1 ∪ Cl ∪ Cl+1 and j ∈ Cl−1 ∩ Cl ∩ Cl+1 . Then, similarly to before we can claim that





FtC in in JkOL − JkOL +1 Ck , C +1 (Ck , C ) = ci (tl ) + c j (tl+1 ) − di (tl ) + d j (tl ) out + cout j (tl ) + ci (tl+1 ) − ci (tl ) − ci (tl+1 ).

If condition (23) does not hold, then

JkOL +1



Ck

, C FtC



>

(26)

JkOL (C , C ), +1 k

from which we conclude that C FtC is not optimal. The thesis is proven for l = k + 1, . . . , k + T − 2. A similar reasoning can be done for the last step l = k + T − 1, with the only difference that there exists no subsequent step.  If we further consider that contents cannot be accessed from the cache while being erased from the cache itself, i.e., cout :=  cout , then the optimality conditions for FtC in (23) and (24) assume a more intuitive form. Corollary 1 (Simplified FtC optimality conditions). If we consider  cout := cout , then FtC is optimal for OL if and only if, for all i ∈ ClFtC and j ∈ / ClFtC ,

ciin (tl ) + cin j (tl+1 ) ≤



tl

tl+1





Nr  pi (t ) −  p j (t ) dt

(27)



Nr  pi (t ) −  p j (t ) dt .

(28) 

We now provide further insights to the FtC optimality conditions. Intuition 1. (Approximated optimality condition) Let us further simplify the expressions for the FtC optimality conditions. If we consider the Taylor expansion ci (tl ) ≈ V (1 + N  pi (tl )/2 ) for  pi  1 of the approximated expression of ciin (tl ) in (14), then we can approximate (27) and (28) as

2V ≤ N V ≤ N



tl+1

tl











r  pi (t ) −  p j (t ) dt −

tk+T

tk+T −1

V  pi (tl ) 2

r  pi (t ) −  p j (t ) dt +

V  pi (tK ) 2

for all i ∈ ClFtC , j ∈ / ClFtC . If moreover the time required to retrieve a content is negligible with respect to the cache update interval, i.e., V/r  tl+1 − tl , then the terms V2  pi (tl ), V2  pi (tK ) also become negligible and the conditions (27) and (28) can be finally approximated as

2V ≤ Nr



tl+1

min

i∈ClFtC , j∈ / ClFtC

tl

( pi (t ) −  p j (t ))dt , l = k + 1, . . . , k + T − 2 (29)

k + T − 2 then we claim that  OL in in JkOL +1 Ck , C − Jk+1 (Ck , C ) ≤ ci (tl ) + c j (tl+1 ) − di (tl ) + d j (tl )



tk+T

V ≤ Nr

 min

i∈ClFtC , j∈ / ClFtC

tk+T



tk+T −1



 pi (t ) −  p j (t ) dt .

(30)

From (29) and (30) we understand that FtC is optimal whenever a combination among the following three events occurs: i) the content size V is sufficiently small to consider the cost to retrieve a popular content as negligible with respect to the resulting traffic gain, ii) the number of users N is sufficiently high for the pull rate to be able to sustain the whole cache update rate r, iii) the popular t

ity gap t l+1  pi (t ) −  p j (t ) dt between the most popular contents l and the remaining ones is sufficiently high to justify the traffic cost to retrieve at each update instant the most popular contents. Intuition 2. From a different perspective, Theorem 1 conveys the message that the FtC strategy is optimal if and only if the cost to swap an unpopular content j with a popular content i (i.e., the left-hand side of (23) and (24)) during the interval [tl ; tl + V/r] is smaller than the core network traffic gain achieved during the successive interval [tl + V/r; tl+1 ] (i.e., the right-hand side of (23) and (24)). In order to provide an easy example that also helps the reader to understand the proof of Theorem 1, let us assume that all contents have the same popularity over the whole time horizon except two of them, contents i and j. Content i is popular only during the time interval [tl ; tl+1 ], i.e., i ∈ ClFtC and i ∈ / CnFtC for all n = l. Conversely, content j is always popular except during the same interval [tl ; tl+1 ], i.e., j ∈ CnFtC for all n = l and j ∈ / ClFtC . Then FtC is optimal if and only if the traffic cost of retrieving content i (= ciin (tl )) and erasing content j (= cout (tl )) at time tl , and re-retrieving j j (= cin (tl+1 )) and erasing i (= ciout (tl+1 )) at time tl+1 , is smaller j than the benefit brought by content i during the interval [tl ; tl+1 ] (= di (tl ) − d j (tl )) in terms of traffic reduction on the core network. 4.1.2. COL optimal cache update under FtC OL strategy Theorem 1 claims that under certain conditions specified in (23) and (24) the FtC strategy is optimal in the OL sub-problem.

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

Under such conditions, the complexity associated to the solution of our main Closed/Open-Loop (COL) problem reduces drastically, as we show in the following. Recall that our main COL problem, which determines the optimal cache update strategy CkCOL at time tk , writes:

CkCOL = arg min Ck ∈S

OL ϕkCL (Ck , CkCOL −1 , Nk ) + Jk+1 (Ck ).

57

greatly reduced to the contents that are either popular in the current interval, or in the next interval, or that have been requested at time tk . In fact, we can rewrite (32) as

CkCOL ≈ arg min Ck ∈S



V 1I(Nk (i )=0) +

i∈Ck \CkCOL −1

+

We now suppose that conditions (23) and (24) hold. Therefore, JkOL (C ) can be calculated under the FtC policy. We first observe +1 k





i∈CkCOL \Ck −1

di (tk ) +

i∈C k

V Nk (i ) 

ciin (tk+1 )

i∈CkFtC \Ck +1

where S  = CkFtC ∪ CkFtC ∪ {i : Nk (i ) ≥ 1}. +1

C FtC

that, since the FtC policy does not depend on the instantaneous cache update Ck , a change in Ck only affects the future openloop cost at the next time step tl+1 . Therefore we can rewrite the COL problem as

CkCOL = arg min Ck ∈S

OL FtC ϕkCL (Ck , CkCOL −1 ) + ϕk+1 (Ck+1 , Ck ),

(31)

where CkFtC are the most popular contents during [tk+1 ; tk+2 ]. We +1 then remark the following. Remark 1 (Limited prediction capability). Under conditions (23) and (24), the prediction of the popularity over only the next two cache update periods is sufficient to compute the optimal instantaneous cache update CkCOL . In fact, from expression in (31) we notice that the optimal cache update CkCOL at time tk only depends on the values of popularity  pi (t ), for all i ∈ M, for all t ∈ [tk ; tk+2 ]. It easily follows that, under the assumption that FtC is optimal for OL, we can further rewrite the optimal COL cache update at time tk as follows. Corollary 2 (COL complexity reduction). Under the assumption that conditions in (23) and (24) hold, the expression of the optimal COL cache update CkCOL at time tk boils down to

CkCOL = arg min Ck ∈S

i∈CkFtC +1



+



ϕkCL (Ck , CkCOL −1 , Nk ) + ciout

ciin (tk+1 ) − ci (tk+1 )

\Ck

(tk+1 ).

i∈Ck \CkFtC +1

 In order to provide further insight to the problem, we also no tice that cin  c, cout . If we also consider cout = cout , we can apCOL proximate Ck as

CkCOL ≈ arg min Ck ∈S





V 1I(Nk (i )=0) +

i∈Ck \CkCOL −1

+

 i∈C k

V Nk (i )

i∈CkCOL \Ck −1

di (tk ) +



ciin (tk+1 ).

(32)

i∈CkFtC \Ck +1

Remark 2 (Replacement/pre-fetching). From the expression in (32) we observe that the optimal COL update policy is a combination between a replacement and a pre-fetching policy. In fact, the decision to store a content at time tk should optimally trade off among i) passively storing a requested content when requested  (even if not popular) through the term V 1I(Nk (i )=0 ), ii) i∈C \C COL k

k−1

proactively retrieve a content being currently popular of the con tent through i∈C di (tk ) and iii) pre-fetching a content predicted k to be popular in the future, during the next cache update interval,  in via i∈C FtC \C ci (tk+1 ). k+1

k

Remark 3 (Search space reduction). From expression (32) we realize that the search space S of the optimal cache update CkCOL can be

4.2. Benefits and robustness of COL optimal policy In this section we show the results of some numerical investigations, highlighting the benefit of taking into account the content popularity predictions, even when those are largely inaccurate, to reduce of traffic generated on the core network via our COL strategy. We still need to choose a suitable benchmark for our COL cache update policy. It is well known that the class of cache pre-fetching policies is superior to the class of cache replacement ones, as LRU and LFU (see, e.g., the survey in [24]). Moreover, our COL policy is a provably optimal combination between pre-fetching and replacement under the condition that future content popularity can be estimated. Therefore, by construction, also our policy is superior to cache replacement policies, that in fact represent a special case of cache update policies where the pre-fetching aspect is absent. For this reason, for our comparison we will focus on the general class of cache update policies, that combine the passive (i.e., replacement) and the pro-active (i.e., pre-fetching) behaviors. Moreover, in order to stress the importance of exploiting content popularity predictions to update the cache, we will compare our COL policy against the best policy that only looks at past popularity, without exploiting future popularity predictions. We call it Store-the-MostPopular (SMP) and, naturally, it updates the cache at time tk with the contents that have been the most popular during the last decision interval [tk−1 ; tk ). In other words, SMP tries to evaluate which contents are the most profitable to be stored in the cache by relying on the most up-to-date popularity estimate at its disposal, i.e., during interval [tk−1 ; tk ). Clearly, if the content popularity is constant over time, then SMP manages to estimate perfectly the most popular contents in the future; hence, in this case SMP performs as well as the COL optimal update policy. On the other hand, when the content popularity is more volatile then our COL strategy has an edge over any cache update strategy that only relies on statistics on past popularity. This trend is confirmed in Fig. 9, where we show the performance gain of our COL update strategy with respect to the classic SMP strategy versus the geometric decay rate of content popularity auto-correlation function (decay rate equal to 0 means white noise, 1 amounts to constant popularity). More in details, we assumed for simplicity that the popularity evolves at the granularity of the decision time interval, i.e., pi (t) remains constant for t ∈ [tk , tk+1 ) and switches to a new value pi (tk+1 ) at t = tk+1 . Then, we considered that the correlation between the content popularity samples pi (tk ) and pi (tk− ) decays in the order of σ  . We then evaluate the robustness of our COL cache update policy with respect to prediction accuracy degradation. The magnitude of popularity prediction error is evaluated as a percentage of the real popularity value. Remarkably, as shown in Fig. 9, COL strategy is fairly robust with respect to prediction errors, as the gain decreases by at most 3% when the prediction error is around 30%. These results clearly advocate the exploitation of content popularity predictions for cache management purposes, since predictions allow to improve the cache update performance even when they are largely inaccurate.

58

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

law,

core network traffic gain [%]

30



gain w.r.t. SMP (perfect prediction) gain w.r.t. SMP (10% prediction error) gain w.r.t. SMP (20% prediction error) gain w.r.t. SMP (30% prediction error)

25

x˙ i (t ) = 0

∀ t ∈ [tk+1 ; tk+T ].

(35)

i∈M

We now find an expression for the objective function. Thanks to Proposition 1 we know that

20

Eφi (xi , x˙ i , t ) = V x˙ i (t )H (V x˙ i (t ))(1 −  pi (t )(1 − xi (t )))N + Nr  pi (t )(1 − xi (t ))

15

(36)

where H is the Heaviside function, i.e., H (x ) = 1 for x ≥ 0 and H (x ) = 0 for x < 0. By putting together equations (34,35,36) we obtain the following optimal control problem:

10 5

JkOL +1 (Ck ) = min



u(t )

tk+1

0 0

0.1

0.2

0.3

0.4

0.5

0.6

decay rate of popularity auto-correlation Fig. 9. Performance gain of our COL optimal update policy with respect to the Store-the-Most-Popular (SMP) strategy, that stores in the cache the contents that have been the most popular in the last time period. We evaluated the gain in terms of traffic generated on the core network, a) for different values of geometric decay rate of content popularity (0 means white noise, 1 means constant popularity) and b) for different prediction error values. We considered C/V M = 1/3.

tk+T



E φi (xi , x˙ i , t ) dt

(37)

i∈M

subject to the constraints:

x˙ i (t ) = (1 − ui (t )) + ui (t ) 

 r  −γ xi (t ) e −1 V

 r

1 − eγ (xi (t )−1) , 1 ≤ i ≤ M, t ∈ [tk+1 ; tk+T ] (38) V ∀ t ∈ [tk+1 ; tk+T ]

x˙ i (t ) = 0,

i∈M

The last section concludes our study on the cache update model at discrete time. In this conclusive section we move on to a generalization of our model in continuous time t and with continuous control variables u(t). More specifically, we will reformulate the open-loop core network traffic minimization sub-problem OL by assuming that the cache can be updated seamlessly, i.e., at all continuous time values t ∈ [tk+1 ; tk+T ]. Moreover, we assume that the rate V x˙ i at which the cache retrieves/erases each content i can be tuned with infinite granularity. Technically speaking, our idea here consists in applying a multiplicative control ui (t) over the derivative x˙ i (t ) at each time t ∈ [tk+1 ; tk+T ] and for all contents i. Therefore, the content allocation over the time xi (t) is the solution of a differential equation involving the term ui (t )x˙ i (t ). Contrarily to the original OL formulation in (4), we here allow for a fluid cache control, where x˙ i (t ) can assume all real values not exceeding the cache update rate r and an erasure rate −r  . Observe in fact that in this scenario we may have r = r. In order to guarantee that xi (t) is bounded between 0 and 1, its derivative must respect certain constraints. Proposition 2. If we consider







r e−γ xi (t ) − 1 ≤ V x˙ i (t ) ≤ r 1 − e

ui (t ) ∈ [0; 1],

(39)

1 ≤ i ≤ M, t ∈ [tk+1 ; tk+T ].

We first notice that the evolution of x˙ i (t ) in (38) is continuously differentiable w.r.t. the state xi and the control ui . We then approximate the Heaviside function H with a continuously differentiable function as H (x ) ≈ 12 (1 + tanh(Bx )), where higher values of B correspond to sharper transitions at x = 0. In this case, the new approximated integrand E φi also becomes continuously differentiable w.r.t. xi and ui . Under such approximation, the continuoustime problem in (37) becomes a classic optimal control problem with equality constraints on functions of the control and state variables (see [7], Ch. 1,3), that can be rewritten in more compact form as

JkOL +1 (Ck ) = min u(t )



tk+T

tk+1



gi (xi (t ), ui (t ), t )dt

i∈M

⎧ l x˙ (t ) = hi (xi , ui , t ), 1 ≤ i ≤ M, t ∈ [tk+1 ; tk+T ] ⎪ ⎪ i ⎨ L ( x, u, t ) = 0, t ∈ [tk+1 ; tk+T ] s.t. u ( t ) = 1 I ( i∈Ck ), ∀i ∈ M ⎪ i k +1 ⎪ ⎩ ui (t ) ∈ [0; 1], 1 ≤ i ≤ M, t ∈ [tk+1 ; tk+T ]. We can then construct the Hamiltonian function H as

 γ (x (t )−1 ) i

then xi (t) ∈ [0; 1] for all t ∈ [tk+1 ; tk+T ].

(33)

H (x, u, λ, μ, t ) =

gi (xi , ui , t ) + λi (t )hi (xi , ui , t )

+ μ(t )L(x, u, t )

Let us now introduce the multiplicative control variable ui (t) ∈ [0; 1]. Then, expression (33) becomes

  r  −γ xi (t ) r

e − 1 + ui (t ) 1 − eγ (xi (t )−1) . V V



i∈M



Proof. We first notice that the constraint −r  ≤ V x˙ i (t ) ≤ r is fulfilled for all t such that xi (t) ∈ [0; 1]. We now have to prove that xi (t) cannot overcome the boundaries. We observe that condition (33) ensures that for xi (t) < 0 the derivative x˙ i (t ) > 0. Similarly, for xi (t) > 1 the derivative x˙ i (t ) < 0. The thesis is proven. 

x˙ i (t ) = (1 − ui (t ))

∀i ∈ M

ui (tk+1 ) = 1I(i∈Ck ),

5. Continuous cache update: a formulation

(34)

We further observe that the cache content cannot exceed its capac ity C, i.e., V i∈M xi (t ) = C at all times t. Then, by the conservation

where λi and μ are time-varying Lagrangian multipliers. We know from the minimum Pontryagin principle that the optimal control u∗ (t ) = [u∗i (t )]i∈M satisfies the following equation:

u∗ (t ) = argmin H (x∗ (t ), u, λ(t ), μ(t ), t ), u

∀ t ∈ [tk+1 ; tk+T ] (40)

where x∗ (t ) = [x∗i (t )]i∈M is the optimal content store evolution associated to u∗ (t). Moreover, the Lagrangian λ satisfies the following differential equation:

λ˙ i (t ) = − ∇xi H (x∗ (t ), u∗ (t ), λ(t ), μ(t ), t )

(41)

λi (tk+T ) = 0, ∀ i ∈ M.

(42)

L. Maggi, J. Leguay / Computer Communications 107 (2017) 49–59

The optimal control u∗ (t) and the associated optimal cache update x∗ (t) can be computed by solving (40) and (41) and the equation x˙ ∗i (t ) = hi (x∗i , u∗i , t ), for all i ∈ M, along with the boundary conditions (39) and (42). This is a classic two-point boundary value problem (TP-BVP), that can be solved via advanced numerical methods (see [18] for a survey). 6. Conclusions Motivated by recent advances in techniques for predicting the evolution of multimedia content popularity, in this paper we show how to best exploit popularity predictions to devise cache update policies. Our main aim is to minimize the traffic at the core network, and we assume that the cache can be updated at pre-defined instants. Our cache update technique is inspired by the control theoretic approach called “Model predictive control”, where at each step an action is taken by considering the predicted evolution of the system. We formulate the cache update problem as a Closed/Open-Loop (COL) optimization problem, that is more tractable than the pure Closed-Loop formulation, having a formidable state space dimensionality. COL takes into account the current content requests and computes the minimum traffic achievable in the future via an Open-Loop (OL) formulation. In the OL sub-problem we assume that the cache will be oblivious to the actual content requests in the future, and only knows them in expected value. OL can be solved via a Dynamic Programming approach, that in its original form still suffers from the curse of dimensionality. We then study a very simple OL cache update strategy, Follow-theCrowd (FtC), that prescribes to compute the minimum traffic generated in the future by considering that the most popular contents are always stored in the cache. We provide in Theorem 1 a necessary and sufficient condition under which FtC is optimal in the OL sub-problem, and show its two-fold beneficial impact on the COL original cache update problem (see Corollary 2). Firstly, COL becomes a simple one-step optimization problem with reduced state space. Secondly, the popularity prediction capabilities required to optimally solve COL are limited to only two cache update intervals. Numerical evaluations show that our approach can lead to considerable gains with respect to the natural Store-the-Most-Popular (SMP) policy that only looks at the past. This holds even when the predictions are largely inaccurate, which clearly advocates the exploitation of predictions for cache management purposes. References [1] M. Ahmed, S. Spagna, F. Huici, S. Niccolini, A peek into the future: predicting the evolution of popularity in user generated content, in: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, ACM, 2013, pp. 607–616.

59

[2] W. Ali, S.M. Shamsuddin, Intelligent client-side web caching scheme based on least recently used algorithm and neuro-fuzzy system, in: Advances in Neural Networks–ISNN 2009, Springer, 2009, pp. 70–79. [3] W. Ali, S.M. Shamsuddin, A.S. Ismail, A survey of web caching and prefetching, Int. J. Adv. Soft Comput. Appl. 3 (1) (2011) 18–44. [4] R. Bellman, On the theory of dynamic programming, Proc. Natl. Acad. Sci. U.S.A. 38 (8) (1952) 716. [5] C. Bernardini, T. Silverston, O. Festor, MPC: popularity-based caching strategy for content centric networks, in: Communications (ICC), 2013 IEEE International Conference on., IEEE, 2013, pp. 3619–3623. [6] D. Bertsekas, Dynamic Programming and Optimal Control. Vol. 1, Athena Scientific Belmont, MA, 1995. [7] A.E. Bryson, Y.-C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Hemisphere, Washington, 1975. [8] G. Chatzopoulou, C. Sheng, M. Faloutsos, A first step towards understanding popularity in youtube, in: INFOCOM IEEE Conference on Computer Communications Workshops, 2010, IEEE, 2010, pp. 1–6. [9] Cisco, 2014, Cisco visual networking index: Forecast and methodology, 2014– 2019, http://www.cisco.com/c/en/us/solutions/collateral/service-provider/ ip- ngn- ip- next- generation- network/white_paper_c11-481360.html. [10] M. Leconte, G. Paschos, L. Gkatzikis, M. Draief, S. Vassilaras, S. Chouvardas, Placing dynamic content in caches with small population, in: IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications, 2016, pp. 1–9. [11] J.G. Lee, S. Moon, K. Salamatian, An approach to model and predict the popularity of online contents with explanatory factors, in: Web Intelligence and Intelligent Agent Technology (WI-IAT), 2010 IEEE/WIC/ACM International Conference on. Vol. 1., IEEE, 2010, pp. 623–630. [12] M.A. Maddah-Ali, U. Niesen, Fundamental limits of caching, IEEE Trans. Inf. Theory 60 (5) (2014) 2856–2867. [13] M.D. Ong, M. Chen, T. Taleb, X. Wang, V. Leung, FGPC: fine-grained popularity-based caching design for content centric networking, in: Proceedings of the 17th ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, ACM, 2014, pp. 295–302. [14] G. Paschos, E. Bas¸ tug˘ , I. Land, G. Caire, M. Debbah, Wireless caching: technical misconceptions and business barriers, 2016, ArXiv preprint arXiv:1602.00173. [15] A. Passarella, A survey on content-centric technologies for the current internet: CDN and P2P solutions, Comput. Commun. 35 (1) (2012) 1–32. [16] H. Pinto, J.M. Almeida, M.A. Gonçalves, Using early view patterns to predict the popularity of youtube videos, in: Proceedings of the sixth ACM International Conference on Web Search and Data Mining, ACM, 2013, pp. 365–374. [17] S.J. Qin, T.A. Badgwell, A survey of industrial model predictive control technology, Control Eng. Pract. 11 (7) (2003) 733–764. [18] A.V. Rao, A survey of numerical methods for optimal control, Adv. Astronaut. Sci. 135 (1) (2009) 497–528. [19] J. Roberts, N. Sbihi, Exploring the memory-bandwidth tradeoff in an information-centric network, Proc. of ITC (2013) 1–9. [20] H. Shen, D. Wang, C. Song, A.-L. Barabási, Modeling and predicting popularity dynamics via reinforced Poisson processes, in: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, AAAI Press, 2014, pp. 291–297. [21] G. Szabo, B.A. Huberman, Predicting the popularity of online content, Commun. ACM 53 (8) (2010) 80–88. [22] A. Tatar, M.D. de Amorim, S. Fdida, P. Antoniadis, A survey on predicting the popularity of web content, J.Internet Serv. Appl. 5 (1) (2014) 1–20. [23] W. Tian, B. Choi, V.V. Phoha, An adaptive web cache access predictor using neural network, in: Developments in Applied Artificial Intelligence, Springer, 2002, pp. 450–459. [24] J. Wang, A survey of web caching schemes for the internet, ACM SIGCOMM Comput. Commun. Rev. 29 (5) (1999) 36–46. [25] A. Wolman, M. Voelker, N. Sharma, N. Cardwell, A. Karlin, H.M. Levy, On the scale and performance of cooperative web proxy caching, ACM SIGOPS Operating Syst. Rev. 33 (5) (1999) 16–31.