- Email: [email protected]
Optimizing the LRU algorithm for web caching
ITC 18 / J. Charzinski, R. Lehnert and P. Tran-Gia (Editors) 9 2003 Elsevier Science B.V. All rights reserved.
191
Optimizing the LRU Algorithm for Web Caching P. Jelenkovi6 a* and A. Radovanovi6 a ~Department of Electrical Engineering, Columbia University, New York, NY 10027, USA We analyze a class of randomized Least-Recently-Used (LRU) cache replacement algorithms under the independent reference model with generalized Zipf's law request probabilities. The randomization was recently proposed for Web caching as a mechanism that discriminates between different document sizes. In particular, a requested document that is not found in the cache either replaces a necessary number of least recently used documents with probability depending on its size or the cache is left unchanged. In this framework, we provide explicit asymptotic characterization of the cache fault probability. Using the derived result we prove that the asymptotic performance of this class of algorithms is optimized when the randomization probabilities are chosen inversely proportional to document sizes. In addition, for this optimized and easy to implement policy, we show that its performance is within a constant factor from the optimal static algorithm. An excellent agreement between our approximation and simulation experiments, even for small cache sizes, further validates our asymptotic results.
1. Introduction Caching is widely recognized as an effective method for improving the efficiency and scalability of multimedia content delivery. It is essentially a process of keeping a smaller collection of frequently requested documents at points in the network where they can be accessed with high speed/low latency. One of the main differences between Web caching and traditional caching in computer systems is that Web documents vary considerably in size depending on the type of information they carry. One of the important aspects of engineering efficient Web caching systems is designing document/file replacement algorithms that achieve high hit ratios and are easy to implement. The cache replacement problem consists of selecting and possibly dynamically updating a collection of frequently accessed documents. The most popular replacement algorithms in practice are based on the Least-Recently-Used (LRU) cache replacement rule that possesses many desirable characteristics, including the high hit ratio, low complexity and flexibility to dynamically adapt to possible changes in request patterns. However, the main disadvantage of the LRU replacement algorithm is its indifference to the variability of document sizes. In order to alleviate this problem, a straightforward, randomized extension of the LRU algorithm was proposed in [12]. In this scheme, if a requested document is not found in the cache, it is either brought into the cache with probability that depends on its size or the cache is left unchanged. In the former case, the new page is accommodated by removing the minimum necessary number of the least recently used documents. In this paper, assuming the independence reference model with generalized Zipf's law request probabilities, we provide an explicit asymptotic characterization of the cache fault probability. Using our asymptotic analysis and Hrlder's inequality we show that the performance of this class of randomized *This work is supported by NSF Grant No. 0092113.
192
algorithms is optimized when the randomization probabilities are selected to be inversely proportional to document sizes. This algorithm, termed LRU-S, was considered in [ 12], but its optimality among the randomized LRU policies was not known. Furthermore, we show that LRU-S is within a constant factor from the optimal static arrangement. Our analytical approach uses probabilistic (average-case) analysis that relies on the well-known connection between the LRU fault probability and Move-To-Front (MTF) search cost distribution, e.g. see [5,7]. The analysis exploits the novel large deviation approach and asymptotic results that were recently developed in [7,8]. For additional references on average case analysis of MTF and LRU algorithms see [5,4,7,8]. In contrast to our probabilistic method, the majority of literature on cache replacement rules for variable document sizes is based on combinatorial (amortized, competitive) techniques. In this context, [2] represents one of the first formal treatments of the caching problem with variable-size documents; this paper proposed a new k-competitive GreedyDual-Size algorithm. For very recent results and references on competitive analysis of deterministic, as well as some randomized, online algorithms the reader is referred to [13,6]. The rest of the paper is organized as follows. In Section 2, we formally introduce the MTF searching algorithm and, using the Poisson embedding technique from [4], derive a representation result for the MTF search cost distribution. Then, in Theorem 2 of Section 3, we present our main asymptotic result that characterizes the MTF search cost distribution for the generalized Zipf's law requests. Using this resuit, we show in Theorem 3 of Section 4 that LRU-S algorithm has asymptotically the best performance within the class of randomized LRU rules. Furthermore, in the same section, we prove that LRU-S policy is within a constant factor from the optimal static arrangement. Finally, in Section 5, we provide numerical experiments that demonstrate excellent agreements between our analytical results and simulations. The concluding remarks are presented in Section 6.
2. Randomized move-to-front algorithm We derive our caching results by exploiting the well-known connection between the LRU caching and MTF searching algorithms that will be formally explained in Section 4. Thus, we proceed with defining a randomized MTF algorithm in this section. Consider a finite list of N documents with M possible sizes, Sl, s 2 , . . . , SM that is dynamically updated as follows. When a document of size sk, 1 < k < M is requested, it is either moved to the front of the list with a positive probability pk or the list is left unchanged with a complementary probability 1 - Pk- In the former case, documents that were in front of the requested item are moved down the list to provide the necessary space at the top of the list. This randomized decision depends only on the size of the currently accessed document and, given the sequence of request sizes, the successive decisions are made independently. Assume that the list of N documents is enumerated as L = { (i, k), 1 < i _< Nk, 1 < k < M}, where Nk is the total number of documents of size s k, N1 q- N2 -+- . . . + N M = N and N = ( N 1 , . . . , N M ) . Request process { (Rn, In) }nt2~= - ~ is a sequence of i.i.d, random variables, where an event {Rn = i, In = k} represents a request for document (i, k) having size sk at time n. We denote request probabilities as IP[Rn = i, In = k] = q~k) and, without loss of generality, assume q~k) >_ q~k) > . . . for all 1 < k < M. The performance quantity of interest for this algorithm is the search cost C'N that represents the total size of all documents in the list that are in front of the requested item. On event {Rn = i, In = k } , CNn is equal to the total size of documents in the list that are in front of document (i, k). Our objective in this paper is to characterize the distribution of the stationary search cost C'N and use this result to estimate the cache fault probability of the corresponding LRU replacement scheme. Clearly, the stationary search cost C N exists since the process of permutations on the elements of the list represents an irreducible and aperiodic finite state Markov chain; for the proof of this existence in a more general setting see Lemma 1 of [8]. Throughout the paper we assume that the search cost process is in stationarity.
193
In order to facilitate the analysis, we use the Poisson embedding technique that was first introduced for the MTF searching problem in [4]. The stationary process {(Rn, In), cN}n~=_c~ is embedded into an independent sequence of Poisson points {Tn,--Cx~ < n < c~) of unit rate. We observe the search cost at time T0 that, without loss of generality, is set to TO = 0. Then, due to the Poisson decomposition theorem, the request process for document (i, k) is Poisson with rate q~k). Similarly, if N~ k) (u, t) denotes the number of requests for document (i, k) in (u, t) that resulted in moving (i, k) to the front of the list, (k) the Poisson decomposition theorem implies that this process is Poisson with rate qi Pk. Next, we define
BJ k) (t) = l [ N J k ) ( - t , 0) > 0], t > 0 with IP[B~k) (t) --- 1] = 1 - e -q!k)pkt, , which indicates whether document (i, k) is moved at least once to the front of the list in ( - t , 0). Therefore, given that document (i, k) is requested at time TO = 0, its search cost is equal to
S~k)(t;N)= ~
stBJt)(t).
(1)
(j,l)r
Note that S~ k) (t; N) is monotonic in t, since it is a nondecreasing function of the counting processes
N}k) (-t, O). Next, let T ~ T (Ro, Io) be the time since the last move of the currently requested item (Ro, Io) to the front of the list and note that its distribution is equal to ~[T(R0, I0) > tlRo = i, Io = k] = e-P~q~k)t.
(2)
Now, we state the following representation result. For the case of the ordinary MTF algorithm, this result was first derived using formal languages in [5] and later reproduced in [4]. Theorem 1 The stationary search cost satisfies C N --d S(Io) RO (T; N), where --d denotes equality in distri-
bution.
Proof: Follows from the same arguments as in Proposition 2.1 of [4] (see also Theorem 1 of [8]). Next, for a fixed number of possible document sizes M, we investigate the behavior of the stationary search cost C N as N ~ c~, where N ~ cx~ means mink Nk ~ cx~. In this regard, consider an infinite (k). sequence of probabilities q~k), i >_ 1, 1 < k < M, such that ~']~M=I~-~i=1 q~k) = 1 and, for all k, qi is R N nondecreasing in i. Now, we define a sequence of request processes { ( n , / nN) ), with ,.,(k)
q}k)
-
"/i,N -- ElM1 EiN=/1 qtt); .
l
.
.
.
l
(3)
Let C N be the corresponding stationary search cost defined by request probability q(k) i,N" Furthermore, consider an infinite list of items accessed by the limiting request process ( R ~ , I ~ ) =
(Rn, In) with probabilities q~k). Then, similarly as in the finite list case, let B~k)(t) be a Bernoulli random variable indicating the event that item (i, k) is moved to the front of the list at least once in [ - t , 0), and define s~k)(t) -- ~ t ~k ) stB(t)(t) The variable T represents the time since the last 9 __ (j,)#(', J 9 move of the currently requested item (R0, I0) with its distribution given by (2). Note that these variables are well defined and a.s. finite. However, it is not clear whether the stationary regime of the search cost process exists for this infinite list. To overcome this technical difficulty, similarly as in Proposition 4.4 of [3] for the case of i.i.d, requests of the same size documents, we pass to the limit over finite lists.
194
Proposition 1 The constructed sequence of stationary search costs C N converges in distribution to C, i.e. as N --~ co
-(Io) (T). C N ~ C =zx SRo
(4)
The following section characterizes the asymptotic behavior of the tail of the searching cost C. In this paper, using the standard notation, for any two real functions a(t) and b(t) and fixed to E 11( U {oc} the expression a(t) ~ b(t) as t --+ to denotes limt~t0 a ( t ) / b ( t ) = 1. Similarly, we say that a(t) > b ( t ) a s t ~ to, if liminft~to a ( t ) / b ( t ) > 1; a(t) < b ( t ) h a s a complementary definition. In addition, throughout the paper H denotes a sufficiently large positive constant. The values of H may differ in different places, for example, H / 2 = H , H 2 = H , H + 1 = H, etc.
3. Main results Our results are derived under the following assumption of generalized Zipf's law distribution. This distribution has been widely used for modeling cache request patterns and, in particular, for capturing the Web access behavior, e.g. see [1,10].
Assumption 1 Let q}k) ,.~ ck/iO,ck > O, 1 <_ k <_ m and q}a) = o(1/ia), m < k <_ M as i -+ oo, c~>l. Theorem 2 Under Assumption 1, lim I~[C > x]x ~-1 K(c~)
(
lla llol
- - (O~-- 1) ~ c l
Pl
/
1/~
11(~ lla ,~a-1 [ Cl m Pm Sin) / 1-1/c~
81-t-'''-t-C
\Pl
" q - ' ' " if"
plm-1/a
'
(5)
where
K(c~) =a ( 11- - )
[F ( 1 - - 1 ) ] ~' c~
(6)
o~
Remarks: (i) The constant K ( a ) is monotonically increasing in c~ with limc,t ~ K(c~) = e"~ ..~ 1.78, where 7 is the Euler constant, and lima,1 K(c~) = 1; this was formally provided in Theorem 3 of [7]. (ii) Clearly, by setting Pk -- 1, sk ------1, the theorem reduces to the traditionally studied LRU system, investigated in Theorem 3 of [7]. Note that, by merging M different classes into one, i.e. having all documents of the same size, and enumerating the documents in a nonincreasing order of their probabilities, easy, but somewhat tedious, algebra shows that the tail of the resulting request distribution is asymptotically / l/a l/a equivalent to i,c1 + - - - + cm )a/((c~ - 1)xa-1). In order to prove the preceding result we use Lemmas 1 and 2 of [7] and Lemma 4 of [8]; for masons of completeness, we state these lemmas in the appendix. Proof: Equation (4) easily implies co
-~(Zo) (T) > x] = ~ IP[C > x] = ~[b'Ro
M
~
-.-(~o) (T) > x IR0 - i, I0 - k] IP[Ro = i, Io = klIP[b'Ro
i=1 k = l cx~ c~
--- fO
~
M
2_.,(qi(k)12pke--q[k)PktI~[S} k) (t) > x] dt, ~'-~"
i--1 k--1
(7)
195
where the last equality follows from the independence of T and (R0, I0). Let S(t)(t) = El~176 B}t)(t) and S(t) = ~ M 1 81S(l) (t), representing the total size of distinct documents that are moved to the front of the list in ( - t , 0). Then, S(t) <_ S} k) (t) + Sk <_ S} k) (t) + s, where s =a maxk Sk. We proceed with proving the lower bound first. From (7) we obtain o0 M c~
IPtC > x] >_ r/ E E (q}k))2pke-q}~)pktIP[S(t) > x + s] dt, Jgexa k = l i = 1 ~
(1+2e) a
1/a
1/~
where ge is chosen according to gc a (r[1-1]) ~a~' with a = 81c 1 Pl
+...
1/a 1/c~ "Jr- 8 m C m Pm .
The
preceding inequality and the monotonicity of S(t) (see the remark after equation (1)) yield c~
P [ C > x]>_ P[S(gexa) > x + s] fg
M
c~
(8)
E i~=1"(qi(k))2pke-q}k)pkt dt. ex~ k = l
From Assumption 1 and Lemma 3 of the appendix, it is easy to show that m
ES(g~x ~) = E E SkS(k)(gexa) ~ (1 + 2e)x as x --+ oc.
(9)
k=l
Furthermore, by Assumption 1, for any e > 0, there exists ie such that for all i _> ie, the request probabilities are bounded as q}k) _< 7a-' m < k _< M, and, therefore, Lemma 3 of the appendix yields ie
(X)
limsupt-UaES (k) (t) <_limsupt -1/a E(1--e--q!k)pkt' t-+ cx)
t-~ oo
)+limsupt-1/a t-+c<)
i=1
E
( 1 - e -tpkC/i~ <
He. 1/a.
i=i~ q-1
Thus, since e can be chosen arbitrarily small, we obtain, for m < k ___M, 1
ES(k)(t) = o(t~) as t --+ cxz.
(10)
Then, the estimates in (9) and (10) yield ES(g~x a) >__ (1 + e)(x + s) for x large enough (x _> xE). Hence, by Lemma 4 of the appendix, for any e > 0 and x large enough P[S(g,x a) >_ x + s] > 1 - e. By replacing this inequality in (8), we obtain for x large enough P [ C > x ] _ > ( 1 - e ) EM _1_ k=l Pk
fff(x)
(2o E(pkq}k)
)2e-qi
(k)Pkt dr.
(11)
exa i = 1
Now, by exploiting Lemma 2 of the appendix and Assumption 1, we infer that for t large enough (t _> to) and alll < k < m i=1
)pkCk,l'~ [ 11 F
2-
t -2+1.
Ce
Now, by replacing (12) into (11), we obtain for x large enough m
P[C > x] _> (1 - e ) 2 E _1 fgC~ (pkck)l/a k=l Pk exa c~
2+1,od
(12)
196
Hence, after computing the integral in the preceding expression, multiplying it with x a - l , taking the limit as x -+ c~, and passing e .1. 0, we conclude liminfIP[C x-+oo
>
x]x ~-1 > (aK(a)l) --
__
~C1/'1/apll/a 81 -Jr''' + cml/apml/a8mJ'~a-1 {/ c1 1/a i-I/~ \Pl
-Jr'''
~a ] pm
q- i l - l / a
"
Due to space limitations the proof of the upper bound is provided in the extended version of this paper [9]. <>
4. The optimal randomized LRU algorithm Consider a universe of a large number of documents N, a subset of which can be placed in an easily accessible location of size x, called cache. Interested parties request these documents by searching the cache first and then, if a document is not found there, the outside universe. When the document is not found in the cache, we say that there was a cache fault or miss. In addition, at the time of a miss, a cache replacement algorithm may replace the necessary number of documents in the cache with a newly retrieved document. The replacements are done in order to reduce the number of future faults. This system models the Web caching environment where the parties that request documents correspond to end-users, the cache represents a local proxy server and the document universe is the entire World Wide Web. The minor difference between this caching system and the traditional computer caching one is that cache updates in the case of faults are only optional. In other words, when a cache fault occurs, the requested document can be retrieved directly from its original storage (server) without impacting the cache content. We assume that the request process is the same as in the case of the randomized MTF algorithm described in Section 2. Similarly, the decisions of the corresponding randomized LRU cache replacement algorithm depend on the currently requested document size sk, 1 < k < M. More specifically, in the case of a cache miss for an item of size Sk, the algorithm places the requested item with probability Pk in the cache or leaves the cache unchanged with the complementary probability (1 - Pk)" the successive randomized decisions are independent. In the case of a replacement, the necessary number of least recently accessed documents are removed from the cache in order to accommodate the newly placed item. We investigate the behavior of the randomized LRU caching fault probability p N (x) using the connection between the randomized LRU caching and MTF searching algorithms. To this end, we can assume, without loss of generality, that documents in the cache are arranged according to an increasing order of their last access times. Furthermore, since the fault probability does not depend on the arrangement of documents that are outside of the cache, they can be arranged in an increasing order of the last access times as well. Thus, the ordered list of documents inside and outside of the cache under the randomized LRU rule behaves the same as the randomized MTF scheme. Clearly, using the notation from Section 2, the stationary cache fault probability satisfies IP[CN > x] _< p N (x) = I~[b'Ro'"(l~ N ) + Sto > x] _< ~[C N > x - s], where s = max k Sk. Now, similarly as in (3), we can construct a family of caching systems indexed by N and show that the limiting cache fault probability P ( x ) = limN--+oop N (x) exists. Furthermore, this probability satisfies I~[C > x] < P ( x ) < ~[C > x - s], which, in conjunction with Theorem 2, implies
Lemma 1 Under Assumption 1, P ( x ) ,-~ P'[C > x] as x --+ cx).
197
The following theorem shows that the optimal performance of the randomized LRU replacement scheme is achieved when randomization probabilities are inversely proportional to document sizes. Let Ps (x) be the fault probability for this particular selection of randomization probabilities. Adopting the notation from [12], we will refer to this policy as LRU-S.
Theorem 3 Under Assumption 1, asymptotically optimal randomized LRU replacement scheme that minimizes the expression in (5) is achieved when Pk = 1/Sk, 1 <_ k < M. Furthermore, for this choice of randomization probabilities, the fault probability satisfies lim x - a + l p s ( x ) = z~c~
(
(aK (-a )1)
1
1__ 1 i)a
1 1 ~ a t - ' ' ' at- C~'3m C~81
(13)
Remark: Note that this algorithm can provide an arbitrarily large relative improvement in comparison to the ordinary LRU scheme. In this regard, by setting Pk = 1 in (5), one obtains the asymptotic result for the traditional LRU. Then, computing the ratio between the obtained constant in (5) and (13), setting Ck -- c and letting (say) Sm --+ oe, one easily calculates this limit to be equal to m, which can be made arbitrarily large. Proof: Minimizing the expression in (5) is equivalent to optimizing the following function f(p)
ZX[1/al/a -- kCl Pl
1/c~ 1/c~ ) a - l /
81 "Jr-'''-'[-C m Pm
8m
el 1--1/c~ at-''' -q'-
am 1--1/a ) " Pm
Pl
Next, define ai =a ,Ci ( 1/a /Pi1-1/C~)l/a and bi =a ,,(c i1/a p i1/ a s i )l-1/a ,1 <_ i <_ M . Then, H61der's inequality implies
I
~I
I
f(p)l/(~ = (a~ + . . . + a~)-~ (b~- + . . . + b "-~m )1--~- __>albl + ' " 1
1
_:
-- C~ 81
i
I
+ ambm
1
(14)
nt - . . . nt- C~nSm g,
where the equality is achieved for a~ -1 = bi, i.e. Pi -- 1/si, 1 < i <__m. For m < i < M , since the performance of the algorithm does not depend on Pi, we can select pi -- 1/si as well. Furthermore, (14) and (5) yield (13). O
4.1. Asymptotically optimal static algorithm In this section we consider a greedy static algorithm that places documents in the cache according to a decreasing order of q}k)/Sk (see page 263 of [11 ]) with Pos(X) being the corresponding fault probability for the cache of size x. Using Assumption 1, we show that this algorithm is asymptotically optimal. The fault probability corresponding to the optimal static placement for the cache of size x will be denoted as Po(z). Without loss of generality assume Sl = min Sk = 1. Let aVb = max(a, b) and aAb = min(a, b). Theorem 4 Under Assumption 1,
Po~(~) Po(X)"~
, 1/a tq +
1/a 1--1/a)a (a-
+ cm ~m
1)x c~-1
as x - + o c .
Remark: In Theorem 5.1 of [ 11 ] it was shown that for any general request distribution and cache of size x, Pos(X) < 2Po(x). Due to space limitations the proof of the theorem is provided in [9]. The following easy consequence of the preceding results shows that the LRU-S algorithm is within a constant factor from the optimal static one.
198
Corollary 1 The asymptotic ratio between the fault probabilities of the LRU-S and optimal static algorithm satisfies l i m x ~ P s ( x ) / P o ( x ) : K ( a ) .
Proof: Follows from Theorems 2, 3 and 4. 4.2. Optimizing the latency of document retrieval In the Web environment, it may be desirable to minimize the expected latency of document retrieval, instead of the cache fault probability. In a first approximation, one can assume that the time to deliver a document from the cache is basically zero, while the transmission time of a document of size sk from its original location depends on its size and is equal to Ck- Then, using the same notation as in Section 2, it can be shown that the stationary limiting (as N ~ e~) expected delay q)(x) satisfies 9
= E
>
---(XO)tT since Jtb'~ , ) + sIo > x} represents the event that the currently requested document (R0,10) is not in the cache. Note that in a different framework, Ck may stand for some other cost of not finding document of type k in the cache. Now, by mimicking the proof of Theorem 2, it is easy to show that, under Assumption 1, / .1. 1/a 2..1/ lim ~ ( x ) x a - l K(c~) / l/a 1/a 1/ot lla ,~a-1 [r 1 Wmt4n a z--,~ -- ( a - 1) ~ ca Pl 8 1 - t - ' ' " - t - C m Pm 8m} | 1-X/a -Jr-''' nt- 1-1/a ) " \Pl Pm
Then, similarly as in Theorem 3, it can be demonstrated that the performance of the randomized LRU scheme is optimized if the probabilities Pk are selected according to Pk = dCk/sk, where d = mini s j / r In addition, this optimized algorithm is asymptotically within a constant K ( a ) from the corresponding optimal static arrangement. Since the derivation of these results involves a straightforward repetition of the preceding analysis, we omit the details. This algorithm that selects the randomization probabilities to be proportional to Ck/Sk was also considered in [12].
5. Numerical experiments In this section we provide two experiments which illustrate the performance benefit of exploiting LRUS replacement scheme instead of the ordinary LRU. In view of Theorem 3 and the remark after it, the ratio between the fault probabilities of these two schemes can be, in general, arbitrarily large. Here, we run the LRU-S replacement scheme for the case of m = 2 and show its significant improvement compared to the ordinary LRU. It is surprising how accurately the approximation works even for relatively small values of the number of documents N and cache sizes x. The initial position of items in the cache is chosen uniformly at random. In both experiments, before conducting measurements, we allow 107 units of warm-up time for the system to reach its steady state. The actual measurement time is set to be 108 units long. In all experiments we measure the fault probability for both LRU-S ("*" symbols) and the ordinary LRU ("o" symbols) replacement policies and cache sizes x = 200j, 1 < j < 9. Furthermore, we present our approximation with the solid line on all figures. The total number of documents in both experiments is set to N = 12000, half of which is of size sl = 1, while the other half is of size s~ = 20. ~2) O.08c/i 1"8 where 1 < i < 6000 and In the first experiment we choose qi 1) = 0.92c/i 1"8, q = , _ _ 1/c = z...,i=l~"60001/i 1"8. The experimental results are presented in Figure (a). Even in this straightforward example, the LRU-S policy performs almost 40% better than the ordinary LRU and is matched very precisely by our approximation.
199
-_1 ~ ~ _ ~ _ _ . _ . . . _
z ~o
El.
o o
t _:
o
i.
vo - - asymptoticformula experiment(LRU-S) o experiment(LRU) L ~ ~ cachesize
~
-
-
asymptoticformula- experiment(LRU-S) experiment
1200
1400
,600
400
600
(LRU)
800
1000
cachesize
1200
1~0
1600
Figure (b): Dependent requests
Figure (a): IID requests
The second experiment analyzes the case where the request process is dependent; although this is not covered by the theoretical results, we conjecture that our results extend to the dependent case, based on recent findings in [8,10]. To this end, we assume that the request process is modulated by a two-state Markov chain taking values in {0, 1 } and having transition probabilities P01 = Pl0 = 0.1. In each of ,.,(1) the two states the requests are conditionally i.i.d." in state O, request probabilities are ~oi
q(2) = 0.08co/i 18 ' 1 < i < 6000 and 1/co Oi ---
--
=
0.92co/il.8
v'6~176176 1/i 18 , while in state 1 , the request process follows A.~i=I
= 0.08cl/i 5 , where 1 < i < 6000 and 1/cl - - - v'6~176176 1/i 5. the distribution qli(1) = 0.92Cl/i5, ,(2) 'ili --A..,i--1 As shown in Figure (b), an excellent agreement between the experimental and conjectured theoretical approximation is apparent. However, this remains to be proved. Additional experimental studies of LRU-S policy, including the trace-driven simulation, and comparisons to related heuristic policies, e.g. GreedyDual-Size, can be found in [ 12].
6. Concluding remarks In this paper we investigated a class of randomized LRU caching algorithms under the independent reference model with empirically observed Zipf's law request probabilities. These low complexity algorithms were recently proposed in [12] to mitigate the effect of variable document sizes on Web caching. For this class of algorithms, we provide an explicit asymptotic characterization of the cache fault probability as the cache sizes grow and, using this asymptotic result, show that they achieve the best performance when the randomization probabilities are selected inversely proportional to document sizes; this algorithm is termed LRU-S. The relative gain of LRU-S in comparison to the ordinary LRU algorithm can be arbitrarily large, as pointed out in the remark after Theorem 3. Thus, the minor increase in implementation complexity is compensated well with potentially high improvement in performance. In addition, we show that LRU-S is within a constant factor from the optimal static algorithm. The results are further validated using simulation that produced surprisingly accurate matches even for small cache sizes. Finally, we would like to point out, that the majority of the Web access traffic is highly correlated and, therefore, the use of the independent reference model may be inappropriate. However, since the analysis of randomized LRU follows closely the one of the ordinary LRU, our recent results from [8,10] suggest that Theorems 2 and 3 can be expected to hold for a large class of semi-Markov modulated request sequences that can capture strong statistical correlation, including the long-range dependence. This conjecture is further supported by the second simulation experiment in the preceding section.
200
7. Appendix In this section we state three lemmas that are used in proving Theorem 2. Lemmas 2 and 3 correspond to Lemmas 1 and 2 of [7], while Lemma 4 is Lemma 4 of [8].
Lemma 2 Assume that qi "~ c/i ~ as i ~ oo, with ~ > I and c > O. Then, as t ~ oo
O0 Z(qi)2e--qit~.a i=1
1 C~F(2_l)g--2"Jt-1-"
'
Ot
where F is the Gamma function.
Lemma 3 Let S(t) = E i c~ = I Bi(t)and qi ,.~ e l i ~ as i -4 co, with a > I and c > O. Then, as t ---r 0o m ( t ) ~=ES(t) ~ F ( a - 1 )
c ~ t ~.
Lemma 4 Let {Bi, i > __ 1 } be a sequence of independent Bernoulli random variables, S E[S]. Then f o r any e > O, there exists O, > O, such that
=
E i =oo I
Bi and m =
~ [ I S - ml > me] < H e -~ REFERENCES 1. v. Almeida, A. Bestavros, M. Crovella, and A. de Oliviera. Characterizing reference locality in the WWW. In Proceedings of the Fourth International Conference on Parallel and Distributed Information Systems, Miami Beach, Florida, December 1996. 2. E Cao and S. Irani. Cost-aware WWW proxy caching algorithms. In Proceedings of the USENIX 1997Annual Technical Conference, Anaheim, California, January 1997. 3. J . A . Fill. Limits and rate of convergence for the distribution of search cost under the move-to-front rule. Theoretical Computer Science, 164:185-206, 1996. 4. J.A. Fill and L. Hoist. On the distribution of search cost for the move-to-front rule. Random structures and algorithms, 8(3):179, 1996. 5. E Flajolet, D. Gardy, and L. Thimonier. Birthday paradox, coupon collector, caching algorithms and selforganizing search. Discrete Applied Mathematics, 39:207-229, 1992. 6. S. Irani. Page replacement with multi-size pages and applications to Web caching. Algorithmica, 33(1):384409, 2002. 7. E R. Jelenkovi6. Asymptotic approximation of the move-to-front search cost distribution and least-recentlyused caching fault probabilities. Annals of Applied Probability, 9(2):430-464, 1999. 8. E R. Jelenkovi6 and A. Radovanovi6. Least-Recently-Used Caching with Dependent Requests. Technical Report EE2002-12-201, Department of Electrical Engineering, Columbia University, New York, August 2002. Theoretical Computer Science, to appear. 9. E R. Jelenkovi6 and A. Radovanovi6. Optimizing LRU Caching for Variable Document Sizes. Technical Report EE2003-05-091, Department of Electrical Engineering, Columbia University, New York. http://www.ee.columbia.edu/~predrag/mypub/LRUsize.ps, December 2002. 10. E R. Jelenkovi6 and A. Radovanovi6. Asymptotic Insensitivity of Least-Recently-Used Caching to Statistical Dependency. In Proceedings o f l N F O C O M 2003, San Francisco, April 2003. 11. B. Moret and H. Shapiro. Algorithms from P to NP: Volume 1 Design and Efficiency. The Benjamin/Cummings Publishing Company, Redwood City, CA, 1991. 12. D. Starobinski and D. Tse. Probabilistic methods for Web caching. Performance evaluation, 46(2-3): 125-137, 2001. 13. N. Young. On-line file caching. Algorithmica, 33(1):371-383, 2002.
191
Optimizing the LRU Algorithm for Web Caching P. Jelenkovi6 a* and A. Radovanovi6 a ~Department of Electrical Engineering, Columbia University, New York, NY 10027, USA We analyze a class of randomized Least-Recently-Used (LRU) cache replacement algorithms under the independent reference model with generalized Zipf's law request probabilities. The randomization was recently proposed for Web caching as a mechanism that discriminates between different document sizes. In particular, a requested document that is not found in the cache either replaces a necessary number of least recently used documents with probability depending on its size or the cache is left unchanged. In this framework, we provide explicit asymptotic characterization of the cache fault probability. Using the derived result we prove that the asymptotic performance of this class of algorithms is optimized when the randomization probabilities are chosen inversely proportional to document sizes. In addition, for this optimized and easy to implement policy, we show that its performance is within a constant factor from the optimal static algorithm. An excellent agreement between our approximation and simulation experiments, even for small cache sizes, further validates our asymptotic results.
1. Introduction Caching is widely recognized as an effective method for improving the efficiency and scalability of multimedia content delivery. It is essentially a process of keeping a smaller collection of frequently requested documents at points in the network where they can be accessed with high speed/low latency. One of the main differences between Web caching and traditional caching in computer systems is that Web documents vary considerably in size depending on the type of information they carry. One of the important aspects of engineering efficient Web caching systems is designing document/file replacement algorithms that achieve high hit ratios and are easy to implement. The cache replacement problem consists of selecting and possibly dynamically updating a collection of frequently accessed documents. The most popular replacement algorithms in practice are based on the Least-Recently-Used (LRU) cache replacement rule that possesses many desirable characteristics, including the high hit ratio, low complexity and flexibility to dynamically adapt to possible changes in request patterns. However, the main disadvantage of the LRU replacement algorithm is its indifference to the variability of document sizes. In order to alleviate this problem, a straightforward, randomized extension of the LRU algorithm was proposed in [12]. In this scheme, if a requested document is not found in the cache, it is either brought into the cache with probability that depends on its size or the cache is left unchanged. In the former case, the new page is accommodated by removing the minimum necessary number of the least recently used documents. In this paper, assuming the independence reference model with generalized Zipf's law request probabilities, we provide an explicit asymptotic characterization of the cache fault probability. Using our asymptotic analysis and Hrlder's inequality we show that the performance of this class of randomized *This work is supported by NSF Grant No. 0092113.
192
algorithms is optimized when the randomization probabilities are selected to be inversely proportional to document sizes. This algorithm, termed LRU-S, was considered in [ 12], but its optimality among the randomized LRU policies was not known. Furthermore, we show that LRU-S is within a constant factor from the optimal static arrangement. Our analytical approach uses probabilistic (average-case) analysis that relies on the well-known connection between the LRU fault probability and Move-To-Front (MTF) search cost distribution, e.g. see [5,7]. The analysis exploits the novel large deviation approach and asymptotic results that were recently developed in [7,8]. For additional references on average case analysis of MTF and LRU algorithms see [5,4,7,8]. In contrast to our probabilistic method, the majority of literature on cache replacement rules for variable document sizes is based on combinatorial (amortized, competitive) techniques. In this context, [2] represents one of the first formal treatments of the caching problem with variable-size documents; this paper proposed a new k-competitive GreedyDual-Size algorithm. For very recent results and references on competitive analysis of deterministic, as well as some randomized, online algorithms the reader is referred to [13,6]. The rest of the paper is organized as follows. In Section 2, we formally introduce the MTF searching algorithm and, using the Poisson embedding technique from [4], derive a representation result for the MTF search cost distribution. Then, in Theorem 2 of Section 3, we present our main asymptotic result that characterizes the MTF search cost distribution for the generalized Zipf's law requests. Using this resuit, we show in Theorem 3 of Section 4 that LRU-S algorithm has asymptotically the best performance within the class of randomized LRU rules. Furthermore, in the same section, we prove that LRU-S policy is within a constant factor from the optimal static arrangement. Finally, in Section 5, we provide numerical experiments that demonstrate excellent agreements between our analytical results and simulations. The concluding remarks are presented in Section 6.
2. Randomized move-to-front algorithm We derive our caching results by exploiting the well-known connection between the LRU caching and MTF searching algorithms that will be formally explained in Section 4. Thus, we proceed with defining a randomized MTF algorithm in this section. Consider a finite list of N documents with M possible sizes, Sl, s 2 , . . . , SM that is dynamically updated as follows. When a document of size sk, 1 < k < M is requested, it is either moved to the front of the list with a positive probability pk or the list is left unchanged with a complementary probability 1 - Pk- In the former case, documents that were in front of the requested item are moved down the list to provide the necessary space at the top of the list. This randomized decision depends only on the size of the currently accessed document and, given the sequence of request sizes, the successive decisions are made independently. Assume that the list of N documents is enumerated as L = { (i, k), 1 < i _< Nk, 1 < k < M}, where Nk is the total number of documents of size s k, N1 q- N2 -+- . . . + N M = N and N = ( N 1 , . . . , N M ) . Request process { (Rn, In) }nt2~= - ~ is a sequence of i.i.d, random variables, where an event {Rn = i, In = k} represents a request for document (i, k) having size sk at time n. We denote request probabilities as IP[Rn = i, In = k] = q~k) and, without loss of generality, assume q~k) >_ q~k) > . . . for all 1 < k < M. The performance quantity of interest for this algorithm is the search cost C'N that represents the total size of all documents in the list that are in front of the requested item. On event {Rn = i, In = k } , CNn is equal to the total size of documents in the list that are in front of document (i, k). Our objective in this paper is to characterize the distribution of the stationary search cost C'N and use this result to estimate the cache fault probability of the corresponding LRU replacement scheme. Clearly, the stationary search cost C N exists since the process of permutations on the elements of the list represents an irreducible and aperiodic finite state Markov chain; for the proof of this existence in a more general setting see Lemma 1 of [8]. Throughout the paper we assume that the search cost process is in stationarity.
193
In order to facilitate the analysis, we use the Poisson embedding technique that was first introduced for the MTF searching problem in [4]. The stationary process {(Rn, In), cN}n~=_c~ is embedded into an independent sequence of Poisson points {Tn,--Cx~ < n < c~) of unit rate. We observe the search cost at time T0 that, without loss of generality, is set to TO = 0. Then, due to the Poisson decomposition theorem, the request process for document (i, k) is Poisson with rate q~k). Similarly, if N~ k) (u, t) denotes the number of requests for document (i, k) in (u, t) that resulted in moving (i, k) to the front of the list, (k) the Poisson decomposition theorem implies that this process is Poisson with rate qi Pk. Next, we define
BJ k) (t) = l [ N J k ) ( - t , 0) > 0], t > 0 with IP[B~k) (t) --- 1] = 1 - e -q!k)pkt, , which indicates whether document (i, k) is moved at least once to the front of the list in ( - t , 0). Therefore, given that document (i, k) is requested at time TO = 0, its search cost is equal to
S~k)(t;N)= ~
stBJt)(t).
(1)
(j,l)r
Note that S~ k) (t; N) is monotonic in t, since it is a nondecreasing function of the counting processes
N}k) (-t, O). Next, let T ~ T (Ro, Io) be the time since the last move of the currently requested item (Ro, Io) to the front of the list and note that its distribution is equal to ~[T(R0, I0) > tlRo = i, Io = k] = e-P~q~k)t.
(2)
Now, we state the following representation result. For the case of the ordinary MTF algorithm, this result was first derived using formal languages in [5] and later reproduced in [4]. Theorem 1 The stationary search cost satisfies C N --d S(Io) RO (T; N), where --d denotes equality in distri-
bution.
Proof: Follows from the same arguments as in Proposition 2.1 of [4] (see also Theorem 1 of [8]). Next, for a fixed number of possible document sizes M, we investigate the behavior of the stationary search cost C N as N ~ c~, where N ~ cx~ means mink Nk ~ cx~. In this regard, consider an infinite (k). sequence of probabilities q~k), i >_ 1, 1 < k < M, such that ~']~M=I~-~i=1 q~k) = 1 and, for all k, qi is R N nondecreasing in i. Now, we define a sequence of request processes { ( n , / nN) ), with ,.,(k)
q}k)
-
"/i,N -- ElM1 EiN=/1 qtt); .
l
.
.
.
l
(3)
Let C N be the corresponding stationary search cost defined by request probability q(k) i,N" Furthermore, consider an infinite list of items accessed by the limiting request process ( R ~ , I ~ ) =
(Rn, In) with probabilities q~k). Then, similarly as in the finite list case, let B~k)(t) be a Bernoulli random variable indicating the event that item (i, k) is moved to the front of the list at least once in [ - t , 0), and define s~k)(t) -- ~ t ~k ) stB(t)(t) The variable T represents the time since the last 9 __ (j,)#(', J 9 move of the currently requested item (R0, I0) with its distribution given by (2). Note that these variables are well defined and a.s. finite. However, it is not clear whether the stationary regime of the search cost process exists for this infinite list. To overcome this technical difficulty, similarly as in Proposition 4.4 of [3] for the case of i.i.d, requests of the same size documents, we pass to the limit over finite lists.
194
Proposition 1 The constructed sequence of stationary search costs C N converges in distribution to C, i.e. as N --~ co
-(Io) (T). C N ~ C =zx SRo
(4)
The following section characterizes the asymptotic behavior of the tail of the searching cost C. In this paper, using the standard notation, for any two real functions a(t) and b(t) and fixed to E 11( U {oc} the expression a(t) ~ b(t) as t --+ to denotes limt~t0 a ( t ) / b ( t ) = 1. Similarly, we say that a(t) > b ( t ) a s t ~ to, if liminft~to a ( t ) / b ( t ) > 1; a(t) < b ( t ) h a s a complementary definition. In addition, throughout the paper H denotes a sufficiently large positive constant. The values of H may differ in different places, for example, H / 2 = H , H 2 = H , H + 1 = H, etc.
3. Main results Our results are derived under the following assumption of generalized Zipf's law distribution. This distribution has been widely used for modeling cache request patterns and, in particular, for capturing the Web access behavior, e.g. see [1,10].
Assumption 1 Let q}k) ,.~ ck/iO,ck > O, 1 <_ k <_ m and q}a) = o(1/ia), m < k <_ M as i -+ oo, c~>l. Theorem 2 Under Assumption 1, lim I~[C > x]x ~-1 K(c~)
(
lla llol
- - (O~-- 1) ~ c l
Pl
/
1/~
11(~ lla ,~a-1 [ Cl m Pm Sin) / 1-1/c~
81-t-'''-t-C
\Pl
" q - ' ' " if"
plm-1/a
'
(5)
where
K(c~) =a ( 11- - )
[F ( 1 - - 1 ) ] ~' c~
(6)
o~
Remarks: (i) The constant K ( a ) is monotonically increasing in c~ with limc,t ~ K(c~) = e"~ ..~ 1.78, where 7 is the Euler constant, and lima,1 K(c~) = 1; this was formally provided in Theorem 3 of [7]. (ii) Clearly, by setting Pk -- 1, sk ------1, the theorem reduces to the traditionally studied LRU system, investigated in Theorem 3 of [7]. Note that, by merging M different classes into one, i.e. having all documents of the same size, and enumerating the documents in a nonincreasing order of their probabilities, easy, but somewhat tedious, algebra shows that the tail of the resulting request distribution is asymptotically / l/a l/a equivalent to i,c1 + - - - + cm )a/((c~ - 1)xa-1). In order to prove the preceding result we use Lemmas 1 and 2 of [7] and Lemma 4 of [8]; for masons of completeness, we state these lemmas in the appendix. Proof: Equation (4) easily implies co
-~(Zo) (T) > x] = ~ IP[C > x] = ~[b'Ro
M
~
-.-(~o) (T) > x IR0 - i, I0 - k] IP[Ro = i, Io = klIP[b'Ro
i=1 k = l cx~ c~
--- fO
~
M
2_.,(qi(k)12pke--q[k)PktI~[S} k) (t) > x] dt, ~'-~"
i--1 k--1
(7)
195
where the last equality follows from the independence of T and (R0, I0). Let S(t)(t) = El~176 B}t)(t) and S(t) = ~ M 1 81S(l) (t), representing the total size of distinct documents that are moved to the front of the list in ( - t , 0). Then, S(t) <_ S} k) (t) + Sk <_ S} k) (t) + s, where s =a maxk Sk. We proceed with proving the lower bound first. From (7) we obtain o0 M c~
IPtC > x] >_ r/ E E (q}k))2pke-q}~)pktIP[S(t) > x + s] dt, Jgexa k = l i = 1 ~
(1+2e) a
1/a
1/~
where ge is chosen according to gc a (r[1-1]) ~a~' with a = 81c 1 Pl
+...
1/a 1/c~ "Jr- 8 m C m Pm .
The
preceding inequality and the monotonicity of S(t) (see the remark after equation (1)) yield c~
P [ C > x]>_ P[S(gexa) > x + s] fg
M
c~
(8)
E i~=1"(qi(k))2pke-q}k)pkt dt. ex~ k = l
From Assumption 1 and Lemma 3 of the appendix, it is easy to show that m
ES(g~x ~) = E E SkS(k)(gexa) ~ (1 + 2e)x as x --+ oc.
(9)
k=l
Furthermore, by Assumption 1, for any e > 0, there exists ie such that for all i _> ie, the request probabilities are bounded as q}k) _< 7a-' m < k _< M, and, therefore, Lemma 3 of the appendix yields ie
(X)
limsupt-UaES (k) (t) <_limsupt -1/a E(1--e--q!k)pkt' t-+ cx)
t-~ oo
)+limsupt-1/a t-+c<)
i=1
E
( 1 - e -tpkC/i~ <
He. 1/a.
i=i~ q-1
Thus, since e can be chosen arbitrarily small, we obtain, for m < k ___M, 1
ES(k)(t) = o(t~) as t --+ cxz.
(10)
Then, the estimates in (9) and (10) yield ES(g~x a) >__ (1 + e)(x + s) for x large enough (x _> xE). Hence, by Lemma 4 of the appendix, for any e > 0 and x large enough P[S(g,x a) >_ x + s] > 1 - e. By replacing this inequality in (8), we obtain for x large enough P [ C > x ] _ > ( 1 - e ) EM _1_ k=l Pk
fff(x)
(2o E(pkq}k)
)2e-qi
(k)Pkt dr.
(11)
exa i = 1
Now, by exploiting Lemma 2 of the appendix and Assumption 1, we infer that for t large enough (t _> to) and alll < k < m i=1
)pkCk,l'~ [ 11 F
2-
t -2+1.
Ce
Now, by replacing (12) into (11), we obtain for x large enough m
P[C > x] _> (1 - e ) 2 E _1 fgC~ (pkck)l/a k=l Pk exa c~
2+1,od
(12)
196
Hence, after computing the integral in the preceding expression, multiplying it with x a - l , taking the limit as x -+ c~, and passing e .1. 0, we conclude liminfIP[C x-+oo
>
x]x ~-1 > (aK(a)l) --
__
~C1/'1/apll/a 81 -Jr''' + cml/apml/a8mJ'~a-1 {/ c1 1/a i-I/~ \Pl
-Jr'''
~a ] pm
q- i l - l / a
"
Due to space limitations the proof of the upper bound is provided in the extended version of this paper [9]. <>
4. The optimal randomized LRU algorithm Consider a universe of a large number of documents N, a subset of which can be placed in an easily accessible location of size x, called cache. Interested parties request these documents by searching the cache first and then, if a document is not found there, the outside universe. When the document is not found in the cache, we say that there was a cache fault or miss. In addition, at the time of a miss, a cache replacement algorithm may replace the necessary number of documents in the cache with a newly retrieved document. The replacements are done in order to reduce the number of future faults. This system models the Web caching environment where the parties that request documents correspond to end-users, the cache represents a local proxy server and the document universe is the entire World Wide Web. The minor difference between this caching system and the traditional computer caching one is that cache updates in the case of faults are only optional. In other words, when a cache fault occurs, the requested document can be retrieved directly from its original storage (server) without impacting the cache content. We assume that the request process is the same as in the case of the randomized MTF algorithm described in Section 2. Similarly, the decisions of the corresponding randomized LRU cache replacement algorithm depend on the currently requested document size sk, 1 < k < M. More specifically, in the case of a cache miss for an item of size Sk, the algorithm places the requested item with probability Pk in the cache or leaves the cache unchanged with the complementary probability (1 - Pk)" the successive randomized decisions are independent. In the case of a replacement, the necessary number of least recently accessed documents are removed from the cache in order to accommodate the newly placed item. We investigate the behavior of the randomized LRU caching fault probability p N (x) using the connection between the randomized LRU caching and MTF searching algorithms. To this end, we can assume, without loss of generality, that documents in the cache are arranged according to an increasing order of their last access times. Furthermore, since the fault probability does not depend on the arrangement of documents that are outside of the cache, they can be arranged in an increasing order of the last access times as well. Thus, the ordered list of documents inside and outside of the cache under the randomized LRU rule behaves the same as the randomized MTF scheme. Clearly, using the notation from Section 2, the stationary cache fault probability satisfies IP[CN > x] _< p N (x) = I~[b'Ro'"(l~ N ) + Sto > x] _< ~[C N > x - s], where s = max k Sk. Now, similarly as in (3), we can construct a family of caching systems indexed by N and show that the limiting cache fault probability P ( x ) = limN--+oop N (x) exists. Furthermore, this probability satisfies I~[C > x] < P ( x ) < ~[C > x - s], which, in conjunction with Theorem 2, implies
Lemma 1 Under Assumption 1, P ( x ) ,-~ P'[C > x] as x --+ cx).
197
The following theorem shows that the optimal performance of the randomized LRU replacement scheme is achieved when randomization probabilities are inversely proportional to document sizes. Let Ps (x) be the fault probability for this particular selection of randomization probabilities. Adopting the notation from [12], we will refer to this policy as LRU-S.
Theorem 3 Under Assumption 1, asymptotically optimal randomized LRU replacement scheme that minimizes the expression in (5) is achieved when Pk = 1/Sk, 1 <_ k < M. Furthermore, for this choice of randomization probabilities, the fault probability satisfies lim x - a + l p s ( x ) = z~c~
(
(aK (-a )1)
1
1__ 1 i)a
1 1 ~ a t - ' ' ' at- C~'3m C~81
(13)
Remark: Note that this algorithm can provide an arbitrarily large relative improvement in comparison to the ordinary LRU scheme. In this regard, by setting Pk = 1 in (5), one obtains the asymptotic result for the traditional LRU. Then, computing the ratio between the obtained constant in (5) and (13), setting Ck -- c and letting (say) Sm --+ oe, one easily calculates this limit to be equal to m, which can be made arbitrarily large. Proof: Minimizing the expression in (5) is equivalent to optimizing the following function f(p)
ZX[1/al/a -- kCl Pl
1/c~ 1/c~ ) a - l /
81 "Jr-'''-'[-C m Pm
8m
el 1--1/c~ at-''' -q'-
am 1--1/a ) " Pm
Pl
Next, define ai =a ,Ci ( 1/a /Pi1-1/C~)l/a and bi =a ,,(c i1/a p i1/ a s i )l-1/a ,1 <_ i <_ M . Then, H61der's inequality implies
I
~I
I
f(p)l/(~ = (a~ + . . . + a~)-~ (b~- + . . . + b "-~m )1--~- __>albl + ' " 1
1
_:
-- C~ 81
i
I
+ ambm
1
(14)
nt - . . . nt- C~nSm g,
where the equality is achieved for a~ -1 = bi, i.e. Pi -- 1/si, 1 < i <__m. For m < i < M , since the performance of the algorithm does not depend on Pi, we can select pi -- 1/si as well. Furthermore, (14) and (5) yield (13). O
4.1. Asymptotically optimal static algorithm In this section we consider a greedy static algorithm that places documents in the cache according to a decreasing order of q}k)/Sk (see page 263 of [11 ]) with Pos(X) being the corresponding fault probability for the cache of size x. Using Assumption 1, we show that this algorithm is asymptotically optimal. The fault probability corresponding to the optimal static placement for the cache of size x will be denoted as Po(z). Without loss of generality assume Sl = min Sk = 1. Let aVb = max(a, b) and aAb = min(a, b). Theorem 4 Under Assumption 1,
Po~(~) Po(X)"~
, 1/a tq +
1/a 1--1/a)a (a-
+ cm ~m
1)x c~-1
as x - + o c .
Remark: In Theorem 5.1 of [ 11 ] it was shown that for any general request distribution and cache of size x, Pos(X) < 2Po(x). Due to space limitations the proof of the theorem is provided in [9]. The following easy consequence of the preceding results shows that the LRU-S algorithm is within a constant factor from the optimal static one.
198
Corollary 1 The asymptotic ratio between the fault probabilities of the LRU-S and optimal static algorithm satisfies l i m x ~ P s ( x ) / P o ( x ) : K ( a ) .
Proof: Follows from Theorems 2, 3 and 4. 4.2. Optimizing the latency of document retrieval In the Web environment, it may be desirable to minimize the expected latency of document retrieval, instead of the cache fault probability. In a first approximation, one can assume that the time to deliver a document from the cache is basically zero, while the transmission time of a document of size sk from its original location depends on its size and is equal to Ck- Then, using the same notation as in Section 2, it can be shown that the stationary limiting (as N ~ e~) expected delay q)(x) satisfies 9
= E
>
---(XO)tT since Jtb'~ , ) + sIo > x} represents the event that the currently requested document (R0,10) is not in the cache. Note that in a different framework, Ck may stand for some other cost of not finding document of type k in the cache. Now, by mimicking the proof of Theorem 2, it is easy to show that, under Assumption 1, / .1. 1/a 2..1/ lim ~ ( x ) x a - l K(c~) / l/a 1/a 1/ot lla ,~a-1 [r 1 Wmt4n a z--,~ -- ( a - 1) ~ ca Pl 8 1 - t - ' ' " - t - C m Pm 8m} | 1-X/a -Jr-''' nt- 1-1/a ) " \Pl Pm
Then, similarly as in Theorem 3, it can be demonstrated that the performance of the randomized LRU scheme is optimized if the probabilities Pk are selected according to Pk = dCk/sk, where d = mini s j / r In addition, this optimized algorithm is asymptotically within a constant K ( a ) from the corresponding optimal static arrangement. Since the derivation of these results involves a straightforward repetition of the preceding analysis, we omit the details. This algorithm that selects the randomization probabilities to be proportional to Ck/Sk was also considered in [12].
5. Numerical experiments In this section we provide two experiments which illustrate the performance benefit of exploiting LRUS replacement scheme instead of the ordinary LRU. In view of Theorem 3 and the remark after it, the ratio between the fault probabilities of these two schemes can be, in general, arbitrarily large. Here, we run the LRU-S replacement scheme for the case of m = 2 and show its significant improvement compared to the ordinary LRU. It is surprising how accurately the approximation works even for relatively small values of the number of documents N and cache sizes x. The initial position of items in the cache is chosen uniformly at random. In both experiments, before conducting measurements, we allow 107 units of warm-up time for the system to reach its steady state. The actual measurement time is set to be 108 units long. In all experiments we measure the fault probability for both LRU-S ("*" symbols) and the ordinary LRU ("o" symbols) replacement policies and cache sizes x = 200j, 1 < j < 9. Furthermore, we present our approximation with the solid line on all figures. The total number of documents in both experiments is set to N = 12000, half of which is of size sl = 1, while the other half is of size s~ = 20. ~2) O.08c/i 1"8 where 1 < i < 6000 and In the first experiment we choose qi 1) = 0.92c/i 1"8, q = , _ _ 1/c = z...,i=l~"60001/i 1"8. The experimental results are presented in Figure (a). Even in this straightforward example, the LRU-S policy performs almost 40% better than the ordinary LRU and is matched very precisely by our approximation.
199
-_1 ~ ~ _ ~ _ _ . _ . . . _
z ~o
El.
o o
t _:
o
i.
vo - - asymptoticformula experiment(LRU-S) o experiment(LRU) L ~ ~ cachesize
~
-
-
asymptoticformula- experiment(LRU-S) experiment
1200
1400
,600
400
600
(LRU)
800
1000
cachesize
1200
1~0
1600
Figure (b): Dependent requests
Figure (a): IID requests
The second experiment analyzes the case where the request process is dependent; although this is not covered by the theoretical results, we conjecture that our results extend to the dependent case, based on recent findings in [8,10]. To this end, we assume that the request process is modulated by a two-state Markov chain taking values in {0, 1 } and having transition probabilities P01 = Pl0 = 0.1. In each of ,.,(1) the two states the requests are conditionally i.i.d." in state O, request probabilities are ~oi
q(2) = 0.08co/i 18 ' 1 < i < 6000 and 1/co Oi ---
--
=
0.92co/il.8
v'6~176176 1/i 18 , while in state 1 , the request process follows A.~i=I
= 0.08cl/i 5 , where 1 < i < 6000 and 1/cl - - - v'6~176176 1/i 5. the distribution qli(1) = 0.92Cl/i5, ,(2) 'ili --A..,i--1 As shown in Figure (b), an excellent agreement between the experimental and conjectured theoretical approximation is apparent. However, this remains to be proved. Additional experimental studies of LRU-S policy, including the trace-driven simulation, and comparisons to related heuristic policies, e.g. GreedyDual-Size, can be found in [ 12].
6. Concluding remarks In this paper we investigated a class of randomized LRU caching algorithms under the independent reference model with empirically observed Zipf's law request probabilities. These low complexity algorithms were recently proposed in [12] to mitigate the effect of variable document sizes on Web caching. For this class of algorithms, we provide an explicit asymptotic characterization of the cache fault probability as the cache sizes grow and, using this asymptotic result, show that they achieve the best performance when the randomization probabilities are selected inversely proportional to document sizes; this algorithm is termed LRU-S. The relative gain of LRU-S in comparison to the ordinary LRU algorithm can be arbitrarily large, as pointed out in the remark after Theorem 3. Thus, the minor increase in implementation complexity is compensated well with potentially high improvement in performance. In addition, we show that LRU-S is within a constant factor from the optimal static algorithm. The results are further validated using simulation that produced surprisingly accurate matches even for small cache sizes. Finally, we would like to point out, that the majority of the Web access traffic is highly correlated and, therefore, the use of the independent reference model may be inappropriate. However, since the analysis of randomized LRU follows closely the one of the ordinary LRU, our recent results from [8,10] suggest that Theorems 2 and 3 can be expected to hold for a large class of semi-Markov modulated request sequences that can capture strong statistical correlation, including the long-range dependence. This conjecture is further supported by the second simulation experiment in the preceding section.
200
7. Appendix In this section we state three lemmas that are used in proving Theorem 2. Lemmas 2 and 3 correspond to Lemmas 1 and 2 of [7], while Lemma 4 is Lemma 4 of [8].
Lemma 2 Assume that qi "~ c/i ~ as i ~ oo, with ~ > I and c > O. Then, as t ~ oo
O0 Z(qi)2e--qit~.a i=1
1 C~F(2_l)g--2"Jt-1-"
'
Ot
where F is the Gamma function.
Lemma 3 Let S(t) = E i c~ = I Bi(t)and qi ,.~ e l i ~ as i -4 co, with a > I and c > O. Then, as t ---r 0o m ( t ) ~=ES(t) ~ F ( a - 1 )
c ~ t ~.
Lemma 4 Let {Bi, i > __ 1 } be a sequence of independent Bernoulli random variables, S E[S]. Then f o r any e > O, there exists O, > O, such that
=
E i =oo I
Bi and m =
~ [ I S - ml > me] < H e -~ REFERENCES 1. v. Almeida, A. Bestavros, M. Crovella, and A. de Oliviera. Characterizing reference locality in the WWW. In Proceedings of the Fourth International Conference on Parallel and Distributed Information Systems, Miami Beach, Florida, December 1996. 2. E Cao and S. Irani. Cost-aware WWW proxy caching algorithms. In Proceedings of the USENIX 1997Annual Technical Conference, Anaheim, California, January 1997. 3. J . A . Fill. Limits and rate of convergence for the distribution of search cost under the move-to-front rule. Theoretical Computer Science, 164:185-206, 1996. 4. J.A. Fill and L. Hoist. On the distribution of search cost for the move-to-front rule. Random structures and algorithms, 8(3):179, 1996. 5. E Flajolet, D. Gardy, and L. Thimonier. Birthday paradox, coupon collector, caching algorithms and selforganizing search. Discrete Applied Mathematics, 39:207-229, 1992. 6. S. Irani. Page replacement with multi-size pages and applications to Web caching. Algorithmica, 33(1):384409, 2002. 7. E R. Jelenkovi6. Asymptotic approximation of the move-to-front search cost distribution and least-recentlyused caching fault probabilities. Annals of Applied Probability, 9(2):430-464, 1999. 8. E R. Jelenkovi6 and A. Radovanovi6. Least-Recently-Used Caching with Dependent Requests. Technical Report EE2002-12-201, Department of Electrical Engineering, Columbia University, New York, August 2002. Theoretical Computer Science, to appear. 9. E R. Jelenkovi6 and A. Radovanovi6. Optimizing LRU Caching for Variable Document Sizes. Technical Report EE2003-05-091, Department of Electrical Engineering, Columbia University, New York. http://www.ee.columbia.edu/~predrag/mypub/LRUsize.ps, December 2002. 10. E R. Jelenkovi6 and A. Radovanovi6. Asymptotic Insensitivity of Least-Recently-Used Caching to Statistical Dependency. In Proceedings o f l N F O C O M 2003, San Francisco, April 2003. 11. B. Moret and H. Shapiro. Algorithms from P to NP: Volume 1 Design and Efficiency. The Benjamin/Cummings Publishing Company, Redwood City, CA, 1991. 12. D. Starobinski and D. Tse. Probabilistic methods for Web caching. Performance evaluation, 46(2-3): 125-137, 2001. 13. N. Young. On-line file caching. Algorithmica, 33(1):371-383, 2002.