- Email: [email protected]
A probabilistic cryptanalytic method for a time-variant permutation generator☆
Information Processing Letters 80 (2001) 67–73
A probabilistic cryptanalytic method for a time-variant permutation generator ✩ Jovan Dj. Goli´c School of Electrical Engineering, University of Belgrade, Bulevar Revolucije 73, 11001 Belgrade, Yugoslavia Received 27 June 2000; received in revised form 28 November 2000 Communicated by J.L. Fiadeiro
Abstract An iterative probabilistic method for reconstructing the initial table of the alleged RC4 keystream generator from a number of known entries and a short given keystream segment is developed. The cryptanalytic algorithm consists of a forward and backward recursive computation of the posterior probabilities of the internal state components given a keystream segment. While maintaining the computational complexity, the new method presents a theoretical and practical improvement of a recently proposed method of a similar type, as less entries of the initial table are required to be known for the algorithm to be successful. 2001 Elsevier Science B.V. All rights reserved. Keywords: Keystream generator; Cryptanalysis; Recursive probabilities; Iterative algorithms
1. Introduction Keystream generators for practical stream cipher applications can generally be represented as autonomous finite-state machines whose initial state and possibly the next-state and output functions are secret-key-dependent. A common type of keystream generators consists of a number of possibly irregularly clocked linear feedback shift registers that are combined by a function with or without memory. Such generators are relatively well explored in the open literature. They can possess desired cryptographic properties, but under certain conditions, they may be vulnerable to various divide-and-conquer attacks in the known plaintext (or ciphertext-only) scenario (for a survey, see [5,2]). A different design approach is taken in a software-oriented keystream generator [6] publicized in [7] and known as RC4 (although a public confirmation is still missing). It is widely used in a number of commercial products and standards including Secure Sockets Layer standard SSL 3.0. Given a parameter n (nominally, n = 8), the internal state of RC4 consists of a balanced table (permutation) of 2n different n-bit words and two pointer n-bit words which, at each time, define the positions of two words in the table to be swapped to produce the table at the next time. The table thus varies slowly in time under the control of itself. At each time, the output of RC4 is an n-bit ✩
This paper was presented at ACISP 2000, Brisbane, Australia, July 2000. E-mail address: [email protected] (J.Dj. Goli´c).
0020-0190/01/$ – see front matter 2001 Elsevier Science B.V. All rights reserved. PII: S 0 0 2 0 - 0 1 9 0 ( 0 1 ) 0 0 1 4 6 - 6
68
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
word which is taken from an appropriate position in the table. The output word is bitwise added to the plaintext word to give the ciphertext word. Standard cryptographic properties are hard to prove for RC4. A linear statistical weakness of the RC4 keystream is both theoretically and experimentally established in [3]. The keystream sequence length needed to detect the weakness is considerably shorter than the period and may even be realistic in high speed applications for n 8. However, as this length is fairly large, the result is more of a theoretical than practical interest. Two cryptanalytic algorithms for reconstructing the initial state of RC4 from a short segment of the keystream are proposed in [4]. One consists of the sequential search through the values of the internal state components (one of the pointers and appropriate entries of the table at successive times) that are consistent with the given keystream segment. Its time complexity is close to the time of searching through the square root of all possible initial states. The other consists of a recursive computation of the posterior probabilities of the internal state components given the corresponding keystream symbols. It can recover the content of the initial table provided a sufficient number of its entries are known (about 60% for n = 8). Its time complexity is about 26n steps consisting of computing the product of appropriate probabilities. The main objective of this paper is to develop an iterative probabilistic method for reconstructing the initial state of RC4 from a short segment of the keystream sequence. A careful probabilistic analysis reveals two main deficiencies of the probabilistic approach from [4]. First, the expressions [4] for the posterior probabilities are just approximations as the two so-called ‘change of state’ and ‘observation of output symbol’ effects should be considered simultaneously rather than separately. Second, apart from the forward recursion of the posterior probabilities, the backward recursion of these probabilities is also required. As a result, less entries of the initial table have to be known for the attack to recover the whole initial table. The resulting algorithm has roughly the same time complexity as the corresponding algorithm [4]. In addition, it is established that a simplified independence assumption gives rise to a faster, yet equally successful algorithm. This is illustrated by experimental results obtained by computer simulations. In Section 2, a more detailed description of RC4 keystream generator is presented. In Section 3, general expressions for the forward and backward recursions of the posterior probabilities are determined. Section 4 contains a description of the corresponding iterative cryptanalytic algorithms and Section 5 is devoted to illustrative experimental results.
2. Description of RC4 We follow the description given in [7]. RC4 is in fact a family of algorithms indexed by a parameter n, which is a positive integer typically recommended to be equal to 8. The internal state of RC4 at time t consists of a table n (permutation) St = (St [l])2l=0−1 of 2n different n-bit words and of two pointer n-bit words it and jt . The n-bit words are treated as binary representations of integers. So, the internal memory size in bits is M = log 2n ! + 2n. Let the output n-bit word of RC4 at time t be denoted by Zt . Let initially i0 = j0 = 0. Then the next-state and output functions of RC4 are for every t 1 defined by it = it −1 + 1,
jt = jt −1 + St −1 [it ],
(1)
St [it ] = St −1 [jt ], St [jt ] = St −1 [it ], Zt = St St [it ] + St [jt ] ,
(2) (3)
where all the additions are modulo 2n . It is assumed that all the words except for the swapped ones remain the same (swapping itself is effective only if it = jt ). The output n-bit word sequence is Z = (Zt )∞ t =1 . The initial table S0 is derived from the secret key string in a way which is not important for our cryptanalysis.
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
69
3. Recursive probabilistic analysis In a probabilistic model where the initial permutation S0 is chosen randomly according to the uniform probability distribution, our objective would ideally be to determine the posterior probability Pr{S0 |Z1T }, where Ztt12 = Zt1 , Zt1 +1 , . . . , Zt2 denotes a segment of length t2 − t1 + 1 of the given keystream sequence. To avoid cumbersome expressions, we keep the same notation for random variables and their values. Thus Pr{x} denotes the probability Pr{X = x}, where X is a random variable. If 2nT 2n !, that is, if T (log 2n !)/n, then S0 is expected to be uniquely determined by Z1T . Computing this probability corresponds to the algorithm [1] for estimating the internal states of a hidden Markov chain and is equivalent to the exhaustive search over all possible S0 . Therefore, instead of the internal state as a whole, we consider its individual components, that is, the individual entries of the table St [i], 0 i 2n − 1, and the pointer jt (the pointer it is known). Our ultimate objective is to determine the posterior probabilities Pr{S0 [i]|Z1T }, 0 i 2n − 1. This can be achieved by considering the posterior probabilities Pr{St [i]|Z1T }, 0 i 2n − 1, and Pr{jt |Z1T }, for 0 t T . The algorithm [1] is not applicable because we deal with the internal state components individually rather than with the whole internal state. The essence of our probabilistic method is to derive the forward recursions for the posterior probabilities Pr St [i]|Z1t , 0 i 2n − 1, and Pr jt |Z1t , for 1 t T , and the backward recursions for the posterior probabilities Pr St [i]|ZtT+1 , 0 i 2n − 1, and Pr jt |ZtT+1 ,
for 0 t T − 1.
Both directions are needed because the underlying independence assumption to be explained below gives rise only to approximate expressions. It is important to deal with ZtT+1 rather than ZtT , because the backward recursions for Pr{St [i]|ZtT } and Pr{jt |ZtT } are much more difficult to derive and compute. Unlike the algorithm [1] where the forward and backward recursions are essentially the same, the forward and backward recursions derived here are different, because the Markov chain property does not hold for the individual internal state components. 3.1. Forward recursions For the forward direction, the next-state and output functions can be put into the form jt = jt −1 + St −1 [it ], it = it −1 + 1, S [j ], i = it , t −1 t St [i] = St −1 [it ], i = jt , S [i], i = i , j , t −1 t t Zt = St St −1 [it ] + St −1 [jt ] .
(4)
(5)
(6)
In view of (4)–(6), it is convenient to perform the following Bayesian decomposition Pr{Z1t −1 } · Pr St [i]|Z1t = Pr{Z1t } jt ,St−1 [it ],St−1 [jt ],St−1 [i]
Pr St [i], Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i], Z1t −1 · Pr jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i]|Z1t −1 ,
(7)
where the summation is over all permissible pointer and entry values involved. The jt pointer can take any possible value, whereas different entries of the table must take different values, because St −1 is a permutation. Note that for convenience the summation is over jt although the conditioning is on jt −1 .
70
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
Eq. (7) gives rise to a recursion by using an approximation that the jt −1 pointer and different entries of St −1 are independent of each other when conditioned on Z1t −1 , except that different entries must take different values. Accordingly, (7) can be approximated as Pr{Z1t −1 } · Pr St [i]|Z1t = Pr{Z1t } jt ,St−1 [it ],St−1 [jt ],St−1 [i]
Pr St [i], Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i], Z1t −1 · Pr jt − St −1 [it ]|Z1t −1 · Pr St −1 [it ]|Z1t −1 · Pr{St −1 [jt ]|Z1t −1}
·
Pr{St −1 [i]|Z1t −1}
Pr{St −1 [jt ] = St −1 [it ]|Z1t −1 } Pr{St −1 [i] = St −1 [it ], St −1 [jt ]|Z1t −1 }
(8)
(where formally 0/0 = 0). It is assumed that the probability of each entry of St −1 when the values of it , jt , and i are not different is included only once. Note that the used approximation of the joint probability of different entries of St −1 , under the independence assumption as in [4], depends on the chosen order of these entries (here it , jt , i). Interestingly, another approximation, which is simpler to compute, proves to be somewhat more successful in the experiments reported in Section 5. Namely, under the simplified independence assumption, the joint probability of different entries can be approximated just as the product of their individual probabilities. Thus, the last line of (8) is then replaced by Pr St −1 [jt ]|Z1t −1 · Pr St −1 [i]|Z1t −1 . The multiplicative term Pr{Z1t −1 }/ Pr{Z1t } is determined by normalization Pr St [i]|Z1t = 1.
(9)
St [i]
Similarly, we can obtain a forward recursion for the jt pointer as Pr{jt |Z1t } =
Pr{Z1t −1} · Pr{Z1t }
Pr jt , Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], Z1t −1 ·
St−1 [it ],St−1 [jt ]
Pr jt − St −1 [it ]|Z1t −1 · Pr St −1 [it ]|Z1t −1 ·
Pr{St −1 [jt ]|Z1t −1}
Pr{St −1 [jt ] = St −1 [it ]|Z1t −1 }
where the multiplicative term Pr{Z1t −1 }/ Pr{Z1t } can be determined by normalization Pr jt |Z1t = 1.
,
(10)
(11)
jt
Note that the conditioning in (10) is on jt −1 . Under the simplified independence assumption, the last line of (10) is replaced by the product Pr jt − St −1 [it ]|Z1t −1 · Pr St −1 [it ]|Z1t −1 · Pr St −1 [jt ]|Z1t −1 . The conditional probabilities in the second lines of (8) and (10), that is, Pr St [i], St St −1 [it ] + St −1 [jt ] = Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i], Z1t −1 , Pr jt , St St −1 [it ] + St −1 [jt ] = Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], Z1t −1
(12) (13)
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
71
can be determined from (4)–(6) by a careful and rather involved examination. The resulting expressions are too complicated to be presented here. 3.2. Backward recursions As the next-state function of RC4 is one-to-one, the next-state and output functions for the backward direction in time can be written as jt = jt +1 − St +1 [jt +1 ], it = it +1 − 1, S [j ], i = it +1 , t +1 t +1 St [i] = St +1 [it +1 ], i = jt +1 , S [i], i = i , j , t +1
t +1
(14)
(15)
t +1
Zt +1 = St +1 St +1 [it +1 ] + St +1 [jt +1] .
(16)
In view of (14)–(16), under similar assumptions, we obtain the following analogs of (7)–(10), respectively: Pr{ZtT+2} Pr St [i]|ZtT+1 = · Pr{ZtT+1}
jt+1 ,St+1 [it+1 ],St+1 [jt+1 ],St+1 [i]
Pr{ZtT+2 } Pr St [i]|ZtT+1 = · Pr{ZtT+1 }
jt+1 ,St+1 [it+1 ],St+1 [jt+1 ],St+1 [i]
Pr St [i], Zt +1|jt +1, St +1 [it +1], St +1 [jt +1], St +1 [i], ZtT+2 · Pr jt +1, St +1 [it +1], St +1 [jt +1], St +1 [i]|ZtT+2 ,
(17)
Pr St [i], Zt +1|jt +1, St +1 [it +1], St +1 [jt +1 ], St +1 [i], ZtT+2 ·
Pr jt +1 |ZtT+2 · Pr St +1 [it +1]|ZtT+2 · Pr{St +1 [jt +1 ]|ZtT+2}
·
Pr{St +1 [i]|ZtT+2}
Pr{St +1 [jt +1] = St +1 [it +1]|Zt +2 1T } Pr{St +1 [i] = St +1 [it +1 ], St +1 [jt +1]|ZtT+2 } Pr{ZtT+2 } Pr jt |ZtT+1 = · Pr{ZtT+1 } St+1 [it+1 ],St+1 [jt+1 ]
,
(18)
Pr jt , Zt +1 |jt + St +1 [jt +1], St +1 [it +1 ], St +1 [jt +1 ], ZtT+2 ·
Pr jt + St +1 [jt +1 ]|ZtT+2 Pr St +1 [it +1]|ZtT+2 ·
Pr{St +1 [jt +1 ]|ZtT+2} Pr{St +1 [jt ] = St +1 [it +1 ]|ZtT+2}
. (19)
Note that the conditioning in (19) is on jt +1 . Under the simplified independence assumption, the last lines of (18) and (19) are modified as the products of the respective probabilities. Note that the Bayesian decompositions in (17)–(19) are carefully chosen so as to make the computations as simple as possible. The analogy with (7)–(10) is considerable, but not complete. The conditional probabilities in the second lines of (18) and (19), that is,
72
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
Pr St [i], St +1 St +1 [it +1] + St +1 [jt +1] = Zt +1 |jt +1, St +1 [it +1], St +1 [jt +1], St +1 [i], ZtT+2 ,
(20)
Pr jt , St +1 St +1 [it +1 ] + St +1 [jt +1 ] = Zt +1 |jt + St +1 [jt +1], St +1 [it +1 ], St +1 [jt +1], ZtT+2
(21)
can be determined from (14)–(16) by a careful and rather involved examination. The resulting expressions are too complicated to be presented here. The multiplicative term Pr{ZtT+2 }/ Pr{ZtT+1} is determined from similar normalization requirements. 4. Iterative algorithms The forward recursion (8) can be used to compute the posterior probabilities Pr{St [i]|Z1t }, 0 St [i] 2n − 1, 0 i 2n − 1, from the previously computed posterior probabilities Pr St −1 [k]|Z1t −1 , 0 St −1 [k] 2n − 1, 0 k 2n − 1, and
Pr jt −1 |Z1t −1 ,
0 jt −1 2n − 1.
Note that the summation is effectively only over the variables jt , St −1 [it ], and St −1 [jt ], as St −1 [i] is in all the cases uniquely determined by the remaining variables and the given values of i, St [i], it , and Zt . Similarly, the forward recursion (10) can be used to compute the posterior probabilities Pr{jt |Z1t }, 0 jt 2n − 1, from the previously computed posterior probabilities Pr{St −1 [k]|Z1t −1}, 0 St −1 [k] 2n − 1, 0 k 2n − 1, and Pr{jt −1 |Z1t −1}, 0 jt −1 2n − 1. For each t, 1 t T , the time complexities of updating the posterior probabilities for St [i] and jt are thus about 25n and 23n steps, respectively, each step consisting of computing the product of the involved probabilities. We thus define the procedure FORWARD that, given the parameters 2n , T , and Z1T , computes the output (terminal) probability distributions Pr{ST [i]|Z1T }, 0 ST [i] 2n − 1, 0 i 2n − 1, and Pr{jT |Z1T }, 0 jT 2n − 1, from the given input (initial) probability distributions Pr{S0 [i]}, 0 S0 [i] 2n − 1, 0 i 2n − 1, and (fixed) Pr{j0 } = δ0,j0 , 0 j0 2n − 1 (where δi,j = 1 if i = j and δi,j = 0 if i = j ). Analogously, the backward recursions (18) and (19) can be used to compute the posterior probabilities Pr{St [i]|ZtT+1}, 0 St [i] 2n − 1, 0 i 2n − 1, and Pr{jt |ZtT+1 }, 0 jt 2n − 1, respectively, from the previously computed posterior probabilities Pr{St +1 [k]|ZtT+2}, 0 St +1 [k] 2n − 1, 0 k 2n − 1, and Pr{jt +1|ZtT+2 }, 0 jt +1 2n − 1. The time complexities are the same as in the forward case. We analogously define the procedure BACKWARD that, given the parameters 2n , T , and Z1T , computes the output (initial) probability distributions Pr{S0 [i]|Z1T }, 0 S0 [i] 2n − 1, 0 i 2n − 1, from the given input (terminal) probability distributions Pr{ST [i]}, 0 ST [i] 2n − 1, 0 i 2n − 1, and Pr{jT }, 0 jT 2n − 1. For control, one can also produce the probability distribution Pr{j0 |Z1T }, 0 j0 2n − 1. The basic iterative algorithm IA is composed of M rounds, each round consisting of the procedure FORWARD followed by the procedure BACKWARD, where the output probability distributions of FORWARD are used as the input probability distributions of BACKWARD, and where the input probability distributions of FORWARD in each round are defined as the output probability distributions of BACKWARD in the previous round. Initially, in the first round, the uniform probability distributions Pr{S0 [i]} = 2−n , 0 S0 [i] 2n − 1, 0 i 2n − 1, are used. The output probability distributions of BACKWARD in the last round represent the soft estimate of the initial table S0 that produced the given keystream segment Z1T . The experiments obtained by computer simulations show that the iterative algorithm in its basic form cannot recover the initial table S0 . However, a modified version in which a given number of entries of S0 , that is, S0 [i], 1 i τ , are known can be successful provided that τ is sufficiently large. In this case, the input probability distributions of FORWARD are adjusted accordingly in each round. If these entries are not known, then, at least in
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
73
theory, they can be guessed either by exhaustive search or by one of the methods from [4]. This is not feasible if τ is large. Note that each guess is easily checked for correctness by comparing the given keystream sequence with the output sequence produced from the reconstructed initial table. The miminal keystream sequence length T required for a successful attack can be estimated as the minimal m such that 2nm 2n ! (Tmin = 5, 11, 24, 50, 103, 211 for n = 3, 4, 5, 6, 7, 8, respectively). Clearly, Tmin < 2n . If τ entries are initially known, then this estimate can be reduced accordingly. The total time complexity of IA is thus about 26n corresponding steps. Instead of the independence assumption, we can also work with the simplified independence assumption. Accordingly, by using the corresponding forward and backward recursions and conditional probabilities, we analogously define the simplified procedures FORWARD and BACKWARD and the simplified iterative algorithm IA . Although the total number of computational steps remains the same, the complexity of most steps is thus reduced.
5. Experimental results The objective of experiments conducted by computer simulations was to verify if the iterative algorithms IA and IA can recover the initial permutation S0 and to investigate the features of these algorithms. In each experiment, the keystream segment Z1T is first generated from a pseudorandomly chosen S0 . Then for both IA and IA , the minimal τ as well as the minimal number of rounds, M, for such τ that are required for successful recovery are found. In most experiments it was assumed that T = 2n and in some cases T had to be reduced. Numerous experiments were performed for n = 3 and n = 4, whereas the results for larger n require more computational power than the one available. For most permutations S0 , τmin = 1 or 2 for n = 3 and τmin = 4 or 5 for n = 4. Accordingly, for n = 8, we may expect that τmin is much smaller than 155 (see [4]). Perhaps surprisingly, in most experiments IA and IA were equally successful (the same τmin ), and in a number of cases IA converged faster than IA (smaller Mmin ). Acknowledgement The author is grateful to Dejan Toši´c for help provided with computer simulations.
References [1] L.R. Bahl, J. Cocke, F. Jelinek, J. Raviv, Optimal decoding of linear codes for minimizing symbol error rate, IEEE Trans. Inform. Theory IT20 (1974) 284–287. [2] J.Dj. Goli´c, Recent advances in stream cipher cryptanalysis, Publications Inst. Math. 64 (78) (1998) 183–204. [3] J.Dj. Goli´c, Linear models for a time-variant permutation generator, IEEE Trans. Inform. Theory IT-45 (1999) 2374–2382. [4] L. Knudsen, W. Meier, B. Preneel, V. Rijmen, S. Verdoolaege, Analysis methods for (alleged) RC4, in: K. Ohta, D. Pei (Eds.), Advances in Cryptology — ASIACRYPT’98, Lecture Notes in Comput. Sci., Vol. 1514, Springer, Berlin, 1998, pp. 327–341. [5] A. Menezes, P. van Oorschot, S. Vanstone, Handbook of Applied Cryptography, CRC Press, Boca Raton, FL, 1997. [6] R.L. Rivest, The RC4 encryption algorithm, RSA Data Security, Inc., March 1992. [7] B. Schneier, Applied Cryptography, Wiley, New York, 1996.
A probabilistic cryptanalytic method for a time-variant permutation generator ✩ Jovan Dj. Goli´c School of Electrical Engineering, University of Belgrade, Bulevar Revolucije 73, 11001 Belgrade, Yugoslavia Received 27 June 2000; received in revised form 28 November 2000 Communicated by J.L. Fiadeiro
Abstract An iterative probabilistic method for reconstructing the initial table of the alleged RC4 keystream generator from a number of known entries and a short given keystream segment is developed. The cryptanalytic algorithm consists of a forward and backward recursive computation of the posterior probabilities of the internal state components given a keystream segment. While maintaining the computational complexity, the new method presents a theoretical and practical improvement of a recently proposed method of a similar type, as less entries of the initial table are required to be known for the algorithm to be successful. 2001 Elsevier Science B.V. All rights reserved. Keywords: Keystream generator; Cryptanalysis; Recursive probabilities; Iterative algorithms
1. Introduction Keystream generators for practical stream cipher applications can generally be represented as autonomous finite-state machines whose initial state and possibly the next-state and output functions are secret-key-dependent. A common type of keystream generators consists of a number of possibly irregularly clocked linear feedback shift registers that are combined by a function with or without memory. Such generators are relatively well explored in the open literature. They can possess desired cryptographic properties, but under certain conditions, they may be vulnerable to various divide-and-conquer attacks in the known plaintext (or ciphertext-only) scenario (for a survey, see [5,2]). A different design approach is taken in a software-oriented keystream generator [6] publicized in [7] and known as RC4 (although a public confirmation is still missing). It is widely used in a number of commercial products and standards including Secure Sockets Layer standard SSL 3.0. Given a parameter n (nominally, n = 8), the internal state of RC4 consists of a balanced table (permutation) of 2n different n-bit words and two pointer n-bit words which, at each time, define the positions of two words in the table to be swapped to produce the table at the next time. The table thus varies slowly in time under the control of itself. At each time, the output of RC4 is an n-bit ✩
This paper was presented at ACISP 2000, Brisbane, Australia, July 2000. E-mail address: [email protected] (J.Dj. Goli´c).
0020-0190/01/$ – see front matter 2001 Elsevier Science B.V. All rights reserved. PII: S 0 0 2 0 - 0 1 9 0 ( 0 1 ) 0 0 1 4 6 - 6
68
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
word which is taken from an appropriate position in the table. The output word is bitwise added to the plaintext word to give the ciphertext word. Standard cryptographic properties are hard to prove for RC4. A linear statistical weakness of the RC4 keystream is both theoretically and experimentally established in [3]. The keystream sequence length needed to detect the weakness is considerably shorter than the period and may even be realistic in high speed applications for n 8. However, as this length is fairly large, the result is more of a theoretical than practical interest. Two cryptanalytic algorithms for reconstructing the initial state of RC4 from a short segment of the keystream are proposed in [4]. One consists of the sequential search through the values of the internal state components (one of the pointers and appropriate entries of the table at successive times) that are consistent with the given keystream segment. Its time complexity is close to the time of searching through the square root of all possible initial states. The other consists of a recursive computation of the posterior probabilities of the internal state components given the corresponding keystream symbols. It can recover the content of the initial table provided a sufficient number of its entries are known (about 60% for n = 8). Its time complexity is about 26n steps consisting of computing the product of appropriate probabilities. The main objective of this paper is to develop an iterative probabilistic method for reconstructing the initial state of RC4 from a short segment of the keystream sequence. A careful probabilistic analysis reveals two main deficiencies of the probabilistic approach from [4]. First, the expressions [4] for the posterior probabilities are just approximations as the two so-called ‘change of state’ and ‘observation of output symbol’ effects should be considered simultaneously rather than separately. Second, apart from the forward recursion of the posterior probabilities, the backward recursion of these probabilities is also required. As a result, less entries of the initial table have to be known for the attack to recover the whole initial table. The resulting algorithm has roughly the same time complexity as the corresponding algorithm [4]. In addition, it is established that a simplified independence assumption gives rise to a faster, yet equally successful algorithm. This is illustrated by experimental results obtained by computer simulations. In Section 2, a more detailed description of RC4 keystream generator is presented. In Section 3, general expressions for the forward and backward recursions of the posterior probabilities are determined. Section 4 contains a description of the corresponding iterative cryptanalytic algorithms and Section 5 is devoted to illustrative experimental results.
2. Description of RC4 We follow the description given in [7]. RC4 is in fact a family of algorithms indexed by a parameter n, which is a positive integer typically recommended to be equal to 8. The internal state of RC4 at time t consists of a table n (permutation) St = (St [l])2l=0−1 of 2n different n-bit words and of two pointer n-bit words it and jt . The n-bit words are treated as binary representations of integers. So, the internal memory size in bits is M = log 2n ! + 2n. Let the output n-bit word of RC4 at time t be denoted by Zt . Let initially i0 = j0 = 0. Then the next-state and output functions of RC4 are for every t 1 defined by it = it −1 + 1,
jt = jt −1 + St −1 [it ],
(1)
St [it ] = St −1 [jt ], St [jt ] = St −1 [it ], Zt = St St [it ] + St [jt ] ,
(2) (3)
where all the additions are modulo 2n . It is assumed that all the words except for the swapped ones remain the same (swapping itself is effective only if it = jt ). The output n-bit word sequence is Z = (Zt )∞ t =1 . The initial table S0 is derived from the secret key string in a way which is not important for our cryptanalysis.
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
69
3. Recursive probabilistic analysis In a probabilistic model where the initial permutation S0 is chosen randomly according to the uniform probability distribution, our objective would ideally be to determine the posterior probability Pr{S0 |Z1T }, where Ztt12 = Zt1 , Zt1 +1 , . . . , Zt2 denotes a segment of length t2 − t1 + 1 of the given keystream sequence. To avoid cumbersome expressions, we keep the same notation for random variables and their values. Thus Pr{x} denotes the probability Pr{X = x}, where X is a random variable. If 2nT 2n !, that is, if T (log 2n !)/n, then S0 is expected to be uniquely determined by Z1T . Computing this probability corresponds to the algorithm [1] for estimating the internal states of a hidden Markov chain and is equivalent to the exhaustive search over all possible S0 . Therefore, instead of the internal state as a whole, we consider its individual components, that is, the individual entries of the table St [i], 0 i 2n − 1, and the pointer jt (the pointer it is known). Our ultimate objective is to determine the posterior probabilities Pr{S0 [i]|Z1T }, 0 i 2n − 1. This can be achieved by considering the posterior probabilities Pr{St [i]|Z1T }, 0 i 2n − 1, and Pr{jt |Z1T }, for 0 t T . The algorithm [1] is not applicable because we deal with the internal state components individually rather than with the whole internal state. The essence of our probabilistic method is to derive the forward recursions for the posterior probabilities Pr St [i]|Z1t , 0 i 2n − 1, and Pr jt |Z1t , for 1 t T , and the backward recursions for the posterior probabilities Pr St [i]|ZtT+1 , 0 i 2n − 1, and Pr jt |ZtT+1 ,
for 0 t T − 1.
Both directions are needed because the underlying independence assumption to be explained below gives rise only to approximate expressions. It is important to deal with ZtT+1 rather than ZtT , because the backward recursions for Pr{St [i]|ZtT } and Pr{jt |ZtT } are much more difficult to derive and compute. Unlike the algorithm [1] where the forward and backward recursions are essentially the same, the forward and backward recursions derived here are different, because the Markov chain property does not hold for the individual internal state components. 3.1. Forward recursions For the forward direction, the next-state and output functions can be put into the form jt = jt −1 + St −1 [it ], it = it −1 + 1, S [j ], i = it , t −1 t St [i] = St −1 [it ], i = jt , S [i], i = i , j , t −1 t t Zt = St St −1 [it ] + St −1 [jt ] .
(4)
(5)
(6)
In view of (4)–(6), it is convenient to perform the following Bayesian decomposition Pr{Z1t −1 } · Pr St [i]|Z1t = Pr{Z1t } jt ,St−1 [it ],St−1 [jt ],St−1 [i]
Pr St [i], Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i], Z1t −1 · Pr jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i]|Z1t −1 ,
(7)
where the summation is over all permissible pointer and entry values involved. The jt pointer can take any possible value, whereas different entries of the table must take different values, because St −1 is a permutation. Note that for convenience the summation is over jt although the conditioning is on jt −1 .
70
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
Eq. (7) gives rise to a recursion by using an approximation that the jt −1 pointer and different entries of St −1 are independent of each other when conditioned on Z1t −1 , except that different entries must take different values. Accordingly, (7) can be approximated as Pr{Z1t −1 } · Pr St [i]|Z1t = Pr{Z1t } jt ,St−1 [it ],St−1 [jt ],St−1 [i]
Pr St [i], Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i], Z1t −1 · Pr jt − St −1 [it ]|Z1t −1 · Pr St −1 [it ]|Z1t −1 · Pr{St −1 [jt ]|Z1t −1}
·
Pr{St −1 [i]|Z1t −1}
Pr{St −1 [jt ] = St −1 [it ]|Z1t −1 } Pr{St −1 [i] = St −1 [it ], St −1 [jt ]|Z1t −1 }
(8)
(where formally 0/0 = 0). It is assumed that the probability of each entry of St −1 when the values of it , jt , and i are not different is included only once. Note that the used approximation of the joint probability of different entries of St −1 , under the independence assumption as in [4], depends on the chosen order of these entries (here it , jt , i). Interestingly, another approximation, which is simpler to compute, proves to be somewhat more successful in the experiments reported in Section 5. Namely, under the simplified independence assumption, the joint probability of different entries can be approximated just as the product of their individual probabilities. Thus, the last line of (8) is then replaced by Pr St −1 [jt ]|Z1t −1 · Pr St −1 [i]|Z1t −1 . The multiplicative term Pr{Z1t −1 }/ Pr{Z1t } is determined by normalization Pr St [i]|Z1t = 1.
(9)
St [i]
Similarly, we can obtain a forward recursion for the jt pointer as Pr{jt |Z1t } =
Pr{Z1t −1} · Pr{Z1t }
Pr jt , Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], Z1t −1 ·
St−1 [it ],St−1 [jt ]
Pr jt − St −1 [it ]|Z1t −1 · Pr St −1 [it ]|Z1t −1 ·
Pr{St −1 [jt ]|Z1t −1}
Pr{St −1 [jt ] = St −1 [it ]|Z1t −1 }
where the multiplicative term Pr{Z1t −1 }/ Pr{Z1t } can be determined by normalization Pr jt |Z1t = 1.
,
(10)
(11)
jt
Note that the conditioning in (10) is on jt −1 . Under the simplified independence assumption, the last line of (10) is replaced by the product Pr jt − St −1 [it ]|Z1t −1 · Pr St −1 [it ]|Z1t −1 · Pr St −1 [jt ]|Z1t −1 . The conditional probabilities in the second lines of (8) and (10), that is, Pr St [i], St St −1 [it ] + St −1 [jt ] = Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], St −1 [i], Z1t −1 , Pr jt , St St −1 [it ] + St −1 [jt ] = Zt |jt − St −1 [it ], St −1 [it ], St −1 [jt ], Z1t −1
(12) (13)
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
71
can be determined from (4)–(6) by a careful and rather involved examination. The resulting expressions are too complicated to be presented here. 3.2. Backward recursions As the next-state function of RC4 is one-to-one, the next-state and output functions for the backward direction in time can be written as jt = jt +1 − St +1 [jt +1 ], it = it +1 − 1, S [j ], i = it +1 , t +1 t +1 St [i] = St +1 [it +1 ], i = jt +1 , S [i], i = i , j , t +1
t +1
(14)
(15)
t +1
Zt +1 = St +1 St +1 [it +1 ] + St +1 [jt +1] .
(16)
In view of (14)–(16), under similar assumptions, we obtain the following analogs of (7)–(10), respectively: Pr{ZtT+2} Pr St [i]|ZtT+1 = · Pr{ZtT+1}
jt+1 ,St+1 [it+1 ],St+1 [jt+1 ],St+1 [i]
Pr{ZtT+2 } Pr St [i]|ZtT+1 = · Pr{ZtT+1 }
jt+1 ,St+1 [it+1 ],St+1 [jt+1 ],St+1 [i]
Pr St [i], Zt +1|jt +1, St +1 [it +1], St +1 [jt +1], St +1 [i], ZtT+2 · Pr jt +1, St +1 [it +1], St +1 [jt +1], St +1 [i]|ZtT+2 ,
(17)
Pr St [i], Zt +1|jt +1, St +1 [it +1], St +1 [jt +1 ], St +1 [i], ZtT+2 ·
Pr jt +1 |ZtT+2 · Pr St +1 [it +1]|ZtT+2 · Pr{St +1 [jt +1 ]|ZtT+2}
·
Pr{St +1 [i]|ZtT+2}
Pr{St +1 [jt +1] = St +1 [it +1]|Zt +2 1T } Pr{St +1 [i] = St +1 [it +1 ], St +1 [jt +1]|ZtT+2 } Pr{ZtT+2 } Pr jt |ZtT+1 = · Pr{ZtT+1 } St+1 [it+1 ],St+1 [jt+1 ]
,
(18)
Pr jt , Zt +1 |jt + St +1 [jt +1], St +1 [it +1 ], St +1 [jt +1 ], ZtT+2 ·
Pr jt + St +1 [jt +1 ]|ZtT+2 Pr St +1 [it +1]|ZtT+2 ·
Pr{St +1 [jt +1 ]|ZtT+2} Pr{St +1 [jt ] = St +1 [it +1 ]|ZtT+2}
. (19)
Note that the conditioning in (19) is on jt +1 . Under the simplified independence assumption, the last lines of (18) and (19) are modified as the products of the respective probabilities. Note that the Bayesian decompositions in (17)–(19) are carefully chosen so as to make the computations as simple as possible. The analogy with (7)–(10) is considerable, but not complete. The conditional probabilities in the second lines of (18) and (19), that is,
72
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
Pr St [i], St +1 St +1 [it +1] + St +1 [jt +1] = Zt +1 |jt +1, St +1 [it +1], St +1 [jt +1], St +1 [i], ZtT+2 ,
(20)
Pr jt , St +1 St +1 [it +1 ] + St +1 [jt +1 ] = Zt +1 |jt + St +1 [jt +1], St +1 [it +1 ], St +1 [jt +1], ZtT+2
(21)
can be determined from (14)–(16) by a careful and rather involved examination. The resulting expressions are too complicated to be presented here. The multiplicative term Pr{ZtT+2 }/ Pr{ZtT+1} is determined from similar normalization requirements. 4. Iterative algorithms The forward recursion (8) can be used to compute the posterior probabilities Pr{St [i]|Z1t }, 0 St [i] 2n − 1, 0 i 2n − 1, from the previously computed posterior probabilities Pr St −1 [k]|Z1t −1 , 0 St −1 [k] 2n − 1, 0 k 2n − 1, and
Pr jt −1 |Z1t −1 ,
0 jt −1 2n − 1.
Note that the summation is effectively only over the variables jt , St −1 [it ], and St −1 [jt ], as St −1 [i] is in all the cases uniquely determined by the remaining variables and the given values of i, St [i], it , and Zt . Similarly, the forward recursion (10) can be used to compute the posterior probabilities Pr{jt |Z1t }, 0 jt 2n − 1, from the previously computed posterior probabilities Pr{St −1 [k]|Z1t −1}, 0 St −1 [k] 2n − 1, 0 k 2n − 1, and Pr{jt −1 |Z1t −1}, 0 jt −1 2n − 1. For each t, 1 t T , the time complexities of updating the posterior probabilities for St [i] and jt are thus about 25n and 23n steps, respectively, each step consisting of computing the product of the involved probabilities. We thus define the procedure FORWARD that, given the parameters 2n , T , and Z1T , computes the output (terminal) probability distributions Pr{ST [i]|Z1T }, 0 ST [i] 2n − 1, 0 i 2n − 1, and Pr{jT |Z1T }, 0 jT 2n − 1, from the given input (initial) probability distributions Pr{S0 [i]}, 0 S0 [i] 2n − 1, 0 i 2n − 1, and (fixed) Pr{j0 } = δ0,j0 , 0 j0 2n − 1 (where δi,j = 1 if i = j and δi,j = 0 if i = j ). Analogously, the backward recursions (18) and (19) can be used to compute the posterior probabilities Pr{St [i]|ZtT+1}, 0 St [i] 2n − 1, 0 i 2n − 1, and Pr{jt |ZtT+1 }, 0 jt 2n − 1, respectively, from the previously computed posterior probabilities Pr{St +1 [k]|ZtT+2}, 0 St +1 [k] 2n − 1, 0 k 2n − 1, and Pr{jt +1|ZtT+2 }, 0 jt +1 2n − 1. The time complexities are the same as in the forward case. We analogously define the procedure BACKWARD that, given the parameters 2n , T , and Z1T , computes the output (initial) probability distributions Pr{S0 [i]|Z1T }, 0 S0 [i] 2n − 1, 0 i 2n − 1, from the given input (terminal) probability distributions Pr{ST [i]}, 0 ST [i] 2n − 1, 0 i 2n − 1, and Pr{jT }, 0 jT 2n − 1. For control, one can also produce the probability distribution Pr{j0 |Z1T }, 0 j0 2n − 1. The basic iterative algorithm IA is composed of M rounds, each round consisting of the procedure FORWARD followed by the procedure BACKWARD, where the output probability distributions of FORWARD are used as the input probability distributions of BACKWARD, and where the input probability distributions of FORWARD in each round are defined as the output probability distributions of BACKWARD in the previous round. Initially, in the first round, the uniform probability distributions Pr{S0 [i]} = 2−n , 0 S0 [i] 2n − 1, 0 i 2n − 1, are used. The output probability distributions of BACKWARD in the last round represent the soft estimate of the initial table S0 that produced the given keystream segment Z1T . The experiments obtained by computer simulations show that the iterative algorithm in its basic form cannot recover the initial table S0 . However, a modified version in which a given number of entries of S0 , that is, S0 [i], 1 i τ , are known can be successful provided that τ is sufficiently large. In this case, the input probability distributions of FORWARD are adjusted accordingly in each round. If these entries are not known, then, at least in
J.Dj. Goli´c / Information Processing Letters 80 (2001) 67–73
73
theory, they can be guessed either by exhaustive search or by one of the methods from [4]. This is not feasible if τ is large. Note that each guess is easily checked for correctness by comparing the given keystream sequence with the output sequence produced from the reconstructed initial table. The miminal keystream sequence length T required for a successful attack can be estimated as the minimal m such that 2nm 2n ! (Tmin = 5, 11, 24, 50, 103, 211 for n = 3, 4, 5, 6, 7, 8, respectively). Clearly, Tmin < 2n . If τ entries are initially known, then this estimate can be reduced accordingly. The total time complexity of IA is thus about 26n corresponding steps. Instead of the independence assumption, we can also work with the simplified independence assumption. Accordingly, by using the corresponding forward and backward recursions and conditional probabilities, we analogously define the simplified procedures FORWARD and BACKWARD and the simplified iterative algorithm IA . Although the total number of computational steps remains the same, the complexity of most steps is thus reduced.
5. Experimental results The objective of experiments conducted by computer simulations was to verify if the iterative algorithms IA and IA can recover the initial permutation S0 and to investigate the features of these algorithms. In each experiment, the keystream segment Z1T is first generated from a pseudorandomly chosen S0 . Then for both IA and IA , the minimal τ as well as the minimal number of rounds, M, for such τ that are required for successful recovery are found. In most experiments it was assumed that T = 2n and in some cases T had to be reduced. Numerous experiments were performed for n = 3 and n = 4, whereas the results for larger n require more computational power than the one available. For most permutations S0 , τmin = 1 or 2 for n = 3 and τmin = 4 or 5 for n = 4. Accordingly, for n = 8, we may expect that τmin is much smaller than 155 (see [4]). Perhaps surprisingly, in most experiments IA and IA were equally successful (the same τmin ), and in a number of cases IA converged faster than IA (smaller Mmin ). Acknowledgement The author is grateful to Dejan Toši´c for help provided with computer simulations.
References [1] L.R. Bahl, J. Cocke, F. Jelinek, J. Raviv, Optimal decoding of linear codes for minimizing symbol error rate, IEEE Trans. Inform. Theory IT20 (1974) 284–287. [2] J.Dj. Goli´c, Recent advances in stream cipher cryptanalysis, Publications Inst. Math. 64 (78) (1998) 183–204. [3] J.Dj. Goli´c, Linear models for a time-variant permutation generator, IEEE Trans. Inform. Theory IT-45 (1999) 2374–2382. [4] L. Knudsen, W. Meier, B. Preneel, V. Rijmen, S. Verdoolaege, Analysis methods for (alleged) RC4, in: K. Ohta, D. Pei (Eds.), Advances in Cryptology — ASIACRYPT’98, Lecture Notes in Comput. Sci., Vol. 1514, Springer, Berlin, 1998, pp. 327–341. [5] A. Menezes, P. van Oorschot, S. Vanstone, Handbook of Applied Cryptography, CRC Press, Boca Raton, FL, 1997. [6] R.L. Rivest, The RC4 encryption algorithm, RSA Data Security, Inc., March 1992. [7] B. Schneier, Applied Cryptography, Wiley, New York, 1996.