- Email: [email protected]
Bit complexity of matrix products
Information Processing North-Holland
38 (1991) 237-242
14 June 1991
Andrzej Lingas Department of Computer Science, Lund University, Box 118, 22100 Lund, Sweden Communicated by W.M. Turski Received 24 September 1990 Revised 11 March 1991
Abstract Lingas, A., Bit complexity of matrix products, Information Processing Letters 38 (1991) 237-242.
Keywords: Computational complexity, matrix multiplication, Boolean and min + matrix products, bit complexity, modular arithmetics
uction
plexity of the min + matrix product in a realistic computation model. Oniy recentiy, Dobosiewicz has slightly improved Fredman’s old upper bound
Since Strassen published his first sub-cubic algorithm for arithmetic matrix multiplication [l], a lot of work in this area has been done [2,11]. The best asymptotic upper bound on the number Ma(n) of arithmetic operations necessary to multiply two n x n matrices is presently 0(n2*376), due to Coppersmith and Winograd [2]. For matrices with polynomial size integer entries this implies an upper bound of the same order up to a log n factor cm the bit complexity of arithmetic matrix product. It follows in particular that the Boolean matrix product of two r? x n matrices with binary entries can be computed in time
the bit decomposing
[1,5,6]. By contrast, there is no essential progress in deriving sub-cubic upper bounds on the com-
arithmetic matrix products over small e Chinese remainder theorem [l].
0020-0190/91/$03.50
0( n”(log’2)n/log
n)1’3)
to 0( n3/ &-ii) in a realistic RAM uniform-cost model 131. In [8], Romani showed how to reduce the problem of min + product to the problem of arithmetic matrix multiplication. The reduction is efficient only for matrices with small integer entries [6,I I]. . . *,._n- E.,,..A,,‘.., , WYC OGliVc In tkiis yapcn compiexity
(0 1991 - Elsevier Science Publishers B.V. (North-Holland)
" a
.wd*.v ..mmPI liti-m uypi
h1\1!vsA VVU~U
t-km vu
of arithmetic matrix product by the later problem into
237
Volume 38. Number 5
INFORMATION
bound is slightly better than the traditional [6,11]) and surprisingly yields an
PROCESSING LETTERS
(see
St( log’2’n/log’4’n)
ic, . J - dl.J 1 g
if - ‘) x d,.Jm
We show that our heuristic approximates the min + product of two matrices with entries in (0, l,..., m } within a factor 1 + l/k using
0(~~(n)g(n22k)~z~(log’2’n
remainder theorem [l], any entry c,,/ of C is in one-to-one correspon e vector ( C I.J
improvement of the well-known upper bound on the bit complexity of the Boolean matrix product (see [l, p. 2431, [5], [6, p. 1541). In the remaining part of the paper, we use this upper bound on arithmetic matrix product and the idea of Romani [8] to derive a good approximation algorithm for the min + product for matrices with nonnecessarily small entries, in the fol!o~jv”liigsense: Let C, D be two matrices with positive integer entries and of the same size. We say that D approximates C within a factor f, f 2 1, if for any pair of the corresponding entries c,.,,d,,,, we have
14 June 1991
od
pp.-d,,/
Therefore,
‘,,Jtmod =
we
)
.
can compute C by computing C, = er hand, for each
P,) C(
%.k mod
P,>( 4.j
mod
PI)
(mod
PI)
k=l
by the definition of C and the laws of modular arithmetic [l]. Let A, = A mod p,, and B, = B mod p,, for 1 < I < q. It follows that Cl = A, X B, for 1 G I G q. Thus, the problem of computing C reduces to the problem of computing the q arithmetic matrix products Cr over the ring Z,,, 1 < I G q, as follows:
1. Find the sequence pl, p2,. . . , pq of primes. 2. Compute the matrices A,, B,, 1 < I < q. 3. For I= 1, . . ., q compute the product C, over
+ log k) log m)
bit operations.
% 4. Compute
C
from
the
residue
matrices
ica-
In order to estimate the bit complexity of arithmetic matrix product of two n x n integer matrices A, B, with entries in (0,. . . , m }, one usually multiplies Ma(n) by Mu(log(nm)). In this way, also an upper bound on the bit complexity of Boolean matrix product by reduction to the arithmetic matrix product over the ring n+, has been derive<: (see [l, p. 2431, [5], [6, p. 1541). e can
p4 be the shortest initial, conLet pi, p2,..., tinuous subsequence of the increasing sequence of prime numbers such that the product of p1 through mn. Next, 1 uct of A and 238
In order to shorten our analysis of the bit complexity of the above method, we define the function g(m) bv g(U) = 2 log U/log’*‘U. Note that g(~)g(“) = G!(u). Hence, we can conclude that the number q of primes in the sequence is 0( g( nm)). Also, by the famous Prime Number Theorem ]4], we have p, = O(log n + log m). Step 1 can be done using the sieve of Eratosthenes for numbers 1 through O(log n + log m ). It takes 0( pi*’ ) divisions of numbers of size O(lo% n + log m), and therefore its bit complexity can be estimated by 0( (log n + log m )1*5Mz4(log’*‘n + log’*‘m)).
iNFORMATION IPROCESSING LETTERS
Volume 38. Number 5
By [4, p. 2911, Step 2, i.e. computing the n2q residua with respect to the CJ primes of size Q(log n + log m), can be done using
14 June 1991
the min + matrix product application, we assume that m can be represented with a polynomial in n number of bits, i.e. r??= 2”O(l).Here, it is sufficient to require
0( n2R4u( g( nm)(log (2)n + log’2’m)) log g(nm)) Ma(n) bit operations. Step 3 takes Ma( n)q arithmetic operations with numbers of size O(log n + log m), and it has A4a( n)g( nm)Mu(log(2)n
+ log’2’m)
bit complexity. Recall that the entries of matrices C,, 1 Q I < q, have size 0( log n + log m ). Therefore, by [ 1, p. 311], the fourth step has
in order to bound the estimate for Step 4 by that for Step 3 asymptotically. Putting everything together, we obtain the following theorem by straightforward calculations. Let A, B be two n x n matrices with integer entries in (0, . . . , m >. Suppose that m = 2’ 1%.?“’ and Ma(n)
0( n2Mu( g( nm)(log “In + logf2’m)) log g( nm )) bit complexity. Note the latter estimate asymptotically bounds from above that on bit complexity of Step 2. By Mu ( n ) = O( n log n logf2’n )
= !2(n2 log n)
= Q( n2 1og”‘n)
or m = 2”!“” and Ma(n)
= !2(n2 log n),
respectively. The arithmetic matrix product of A and B can be computed using 0( Ma(n)g(nm)Mu(log
(see [lo] or [l, pp. 270-2741) and
log n + log log m))
bit operations. g( nm ) = 0( logf2’n + log’2’m), we can transform 0(n2g(nm)Mu(log
it to (“n + log’2)m) log g( nm
)) .
In turn, the above estimate can be asymptotically bounded from above by that on bit complexity of Step 3 if we assume a reasonable lower bound on Ma(n) and a reasonable upper bound on m in terms of n. Having in mind further applications of our method for arithmetic matrix product to computing the Boolean product and the min + matrix product, we respectively distinguish two cases. In the case related to the Boolean matrix product application, we assume that m can be represented with a poly-logarithmic in H number of bits, i.e. m = 20% nP’i* Then, it is enough to assume Ma(n)
= st( n2 logt2’n)
By reducing the Boolean matrix product to an arithmetic one over the ring Z,,, , and using the above theorem for m = n and Ma(n)
= Q(n2 log’2’n),
we can improve the upper bound on bit complexity of the Boolean matrix product given in [ 1, p. 2431 by an Q ( log(2)n/log(4)n ) factor (see also [6, p. 1541).
Ma(n)
= Stjn’ logt2)n),
the Boolean marrix product of two n X n matrices takes O( Ma ( n ) log n log’3’r? log’4)n)
in order to bound the estimate for Step 4 by that for Step 3 asymptotically. In the case related to
bit operations. 239
By Theorem 5.7 in [l], we obtain following corollary.
also the
Corollary 2.3. Assuming Ma(n)
= a( nz log’2’n),
the transitive closure of a Boolean n X n matrix can be computed using 0( Ma ( n ) log n log’3’n logt4’n)
3. 5it complexity of min + matrix product In [B], Romani presented the following algorithm for computing the min + product C of two n X n integer matrices A, B by reduction to an arithmetic matrix product. Algorithm 2.
Set m’ IC:1 + /log n]. Transform A and B to matrices A’ and B’ where a,:, = 2-“‘“~4, and b,;, = 2-m% for 1
log c;,,
roof (sketch). To estimate the bit complexity of the min + product, we follow Mehlhorn’s analysis of Romani’s algorithm [6], performing the substitutions mentioned in the above. By [6], the first step takes 0( n2m log n ) bit operations, and the third one
J For the proof of correctness of Romani’s algorithm see [6] or [B]. In [6], Mehlhorn estimated the bit complexity of Romani’s method of computing the rnin + matrix product. For tutorial reasons, he chose Strassen’s O(n@:) bound for arithmetic matrix multiplication, multiplying it by a naive quadratic bound for integer multiplication to derive the estimate. If we use instead the upper bound given in Theorem 2.1 and the term Mu(n), we will obtain the following theorem. The problem 0:’ computing the min + product of two n x n matrices with integer entries in (0, l,..., m) takes Q( Ma(n)g(n22m)Mu(log’2’n
In order to use the method of Theorem 2.1 to implement the second step of Romani’s algorithm, we rescale the entries of A’ and B’ by multiplying them with 2”‘“. Then, we compute the arithmetic product of the so transformed A’ and B’ using the method of Theorem 2.1, and divide the entries of the resulting product back by 22m’m. The additional multiplications and divisions can be realized by simple shifts, and take O(n’m log n) time. The most costly in our implementation of the second step is computing the arithmetic matrix product which takes 0( Ma(n)g(n
X n2”)Mu(log’2’n
bit operations
by Theorem 2.1.
4. Approximating
1
for 1
240
bit operations provided that Ma( n ) = Q( n ‘log n ) and m = no(‘).
0( n2Mu(log m + logt2’n)).
bit operations.
i
14June 1991
INFORMATIONPROCESSINGLETTERS
Volume38, Number 5
+ log m))
+ log m)) q
the min + lnatrix product
Our algorithm for approximating the min + product relies on the efficient method of computing the min + product for matrices with small integer entries estimated in Theorem 3:l. Algorithm 3.
Input:
n X n matrices A, B with positive integer entries, a positive integer approximation parameter k; Output: an n x n matrix D that approximates the min + product of A and B within a factor 1 + l/k; 1 Initialize a Boolean n x n matrix
E by setting all its entries to fake (setting an entry e,,, to true will mean that the value of c,.~ is determined);
Volume 38, Number 5
INFORMATION
PROCESSING
2. Set m to the maximum value of an entry in A or B, and set d to mJ4k. 3. while m > 2k do begin m”+m+2d;
Transform A and B to matrices A’ and B’ by replacing each entry a,,i in A, and b,., in B which is smaller than m’ with [a,,.,/dj and 1b,.,/d 1, respectively, and replacing each entry greater than m’ with [m/d]; Compute the min + product C’ of A’ and B’; for all entries c,:, do if 7e, , and d X c,:, b m/2 be&n d ‘.J + d x ci’.,; e ‘.J + true; end
LETTERS
14 June 1991
while block that d,.! >, mJ2’. bq.J G m + 4d. Otherwise, 2d 1a,,,/M
] + 2d 1bq, ,/2d
’
d,.J=d[a,,,/d]
+d[bq,J/d(-
a
This combined with + d X b,+,/d
yields c,,J < di .j + 2d - 2. On the other hand, we have
m + m/2; d + d/2; end 4. Transform
A and B to matrices A’ and B’ by replacing each entry a,,/ in A, and b,,, in B which is greater than m + d with m + d; Compute the min + product C’ of A’ and B’; for all entries c,:~ do if ,e,, J then d, .J + c,‘J
In the following lemma, we prove rectness of Algorithm 3.
1 > m,/2’-
and d,,/ is determined in an earlier iteration. We conclude that neither a, cl nor bq,J is greater than m’= m + 2d. In this way we obtain
a,,, + b,.j d d X a,,,/d then
Wz have aiso a, 4 +
which implies di J < Ci.J + 2d - 2. We can estimate 2d - 2 by 2m,/i4k2’-‘) which is not greater than m/2 k. Moreover, we have also di. j >, m/2. It follows that 2d - 2 < di.J/k, and in result 1c,., d,,, 1 < d,, //k. Finally, if d,, J is determined in Step 4, d,. J = c,:, and consequently d,_, = a;,q + 6’ . Note that neither a,‘q nor bi,J is greater than 6’; d since otherwise d,,, was determined in Step 3. Thus, a& = a,.4 and bi, J = bq, J which implies d,.J
the cor-
Lemma 4.1. The matrix
D ap_proximates the min -iproduct of the matrices A and B within a factor 1 + l/k.
=
‘,.J’
q
In the next lemma, we analyse the bit complexity of Algorithm 3. mma 4.2. Assume that k = no(‘) and
Ma(n)
= Q(n2 log N).
Algorithm 3 can be implemented using roof. Let 1 < i < n and 1
0( Ma( n)g( n22k) Mu(log”‘n
+ log k) log m)
bit operations.
,.,=min(a,,,+b,,,ll
Let C,,j = a,., + b,,J, and c,:~ = a,‘q + bi,J. Suppose first that d,,, is determined in Step 3. Let I be the number of the iteration of the while block in which d, / is determined, and let m, stand for the maximum value of an entry in A or B. By the definition of the while block, we have m = m,/2’-’ and d = m,/(4k2’-‘). Ii follows by the test in the
oaf. Step 1,2 can be easily implemented using O(n’log m) bit operations. Let us estimate bit
complexity of the block under the while statement. Note that to compute a,, ,/d and b,_J/d it suffices to chop off the lost 11~; m/4k] bits from the binary representation of a,, J and 6,. /, respectively. Therefore, computing d,m’, transforming A and B into A’ and B’, given C’ partially filling D, all 241
INFORMATION
Volume 38. Number 5
PROCESSING
this can be done performing 0( n’log m ) bit operations. The most costly here is computing C’. Note that the entries in A’ and B’ are of size 0( m ’ ) = 0( k ) (also in Step 4). It follows from Theorem 3.1 that computing C’ takes o( Ma( n)g( n22”) Mu(log’%
to derive a large part of our results in the model of log-space uniform have polylog depth and polyno words, in terms of NC algorithms. Of course, this would require more precise definition of Ma(n) u( n ) in terms of uniform circuits. and
+ log ic ))
steps. Putting everything together and observing that the block under the while statement is iterated O(log m) times, we obtain the total estimate on bit complexity of Algorithm 3. 0 Combining Lemma 4.1 with Lemma 4.2, we obtain the following theorem. Suppose that k = no(‘) and Ma(n) = C!2(n’log n ). The min + product of two n X n matrices with entries in (0, i q. . . 9 m >, can be approximated within a factor 1 + l/k using 0( Ma(n)g(n22”)Mu(log
log n + log k) log m)
I would like to express my appreciation to Rolf Karlsson and an unknown referee for valuable comments, and to Ian Munro for informing me about the state of the art in the min + matrix product problem.
eferences
111A.V.
bit operations.
VI
Aho, J.E. Hopcroft and J.D. Ullman, The Design and Analysis of Cornpurer Algorithms (Addison-Wesley, Reading, MA, 1974). D. Coppersmith, S. Winograd, Matrix multiplication via arithmetic progressions, in: Proc. 19th Ann. ACM Symp. on Theory of Computing (1987) l-6.
[31 W. Dobosiewicz,
Iterating our approximation heuristic for the n-tin + matrix product unfortunately does not yield an efficient approximation method for the matrix closure in the sense of the n-tin + operations. Simply, if the matrix entries are of size Q(n), then k has to be set to 2’(“) in order to achieve a constant approximation factor. This increases the bit complexity of the heuristic to at least cubic which makes the approximation solution unattractive compared with the exact solutions to the closure problem having about the same bit complexity [1,3]. The best, presently-known upper bounds on Ma( n ) and Mu( n ) are achieved by algorithms that can be implemented by uniform aritkmetic circuits which have depth O(log2n) [2] or uniform Boolean circuits which have depth O(log n) [9,10], respectively. Therefore, it would also be possible
242
14 June 199i
LETTERS
plus multiplication.
A more efficient algorithm for the minInternat. J. Comput. Math. 32 (1990)
49-60.
141U. Dudley, iSI
Elementary Number Theory (Freeman, New York, 1970). M.J. Fischer and A.R. Meyer, Boolean matrix multiplication and transitive closure, in: IEEE 12th Ann. Symp. on Switching
and Automata Theory (1971) 129-131. Data Structures and AIgorithms 2: Graph Algorithms and NP-Completeness (Springer, Berlin, 1984)
161K. Mehlhom, 155-158.
171I. Munro, Efficient determination
of the transitive closure of a directed graph, Inform. Process. Left. 1 (1971) 56-58.
PI F. Romani, matrix
Shortest-path problem is not harder than multiplication, Inform. Process. Left. 11 (1980)
134-136.
Pi J&i. Reif and S.R. Tate, Optimal
size integer
division
circuits, in: Proc. 2lst Ann. ACM Symp. on Theory of Compt.ting (1989) 264-273. PO1 A. Schijnhage and V. Strassen, Schnelle Multiplikation Gross Zahlen, Comprdng 7 (1971) 281-292.
1111Y. Pan, How to Multiply Matrices Faster, Lecture Notes in Computer
Science
179 (Springer, Berlin).
38 (1991) 237-242
14 June 1991
Andrzej Lingas Department of Computer Science, Lund University, Box 118, 22100 Lund, Sweden Communicated by W.M. Turski Received 24 September 1990 Revised 11 March 1991
Abstract Lingas, A., Bit complexity of matrix products, Information Processing Letters 38 (1991) 237-242.
Keywords: Computational complexity, matrix multiplication, Boolean and min + matrix products, bit complexity, modular arithmetics
uction
plexity of the min + matrix product in a realistic computation model. Oniy recentiy, Dobosiewicz has slightly improved Fredman’s old upper bound
Since Strassen published his first sub-cubic algorithm for arithmetic matrix multiplication [l], a lot of work in this area has been done [2,11]. The best asymptotic upper bound on the number Ma(n) of arithmetic operations necessary to multiply two n x n matrices is presently 0(n2*376), due to Coppersmith and Winograd [2]. For matrices with polynomial size integer entries this implies an upper bound of the same order up to a log n factor cm the bit complexity of arithmetic matrix product. It follows in particular that the Boolean matrix product of two r? x n matrices with binary entries can be computed in time
the bit decomposing
[1,5,6]. By contrast, there is no essential progress in deriving sub-cubic upper bounds on the com-
arithmetic matrix products over small e Chinese remainder theorem [l].
0020-0190/91/$03.50
0( n”(log’2)n/log
n)1’3)
to 0( n3/ &-ii) in a realistic RAM uniform-cost model 131. In [8], Romani showed how to reduce the problem of min + product to the problem of arithmetic matrix multiplication. The reduction is efficient only for matrices with small integer entries [6,I I]. . . *,._n- E.,,..A,,‘.., , WYC OGliVc In tkiis yapcn compiexity
(0 1991 - Elsevier Science Publishers B.V. (North-Holland)
" a
.wd*.v ..mmPI liti-m uypi
h1\1!vsA VVU~U
t-km vu
of arithmetic matrix product by the later problem into
237
Volume 38. Number 5
INFORMATION
bound is slightly better than the traditional [6,11]) and surprisingly yields an
PROCESSING LETTERS
(see
St( log’2’n/log’4’n)
ic, . J - dl.J 1 g
if - ‘) x d,.Jm
We show that our heuristic approximates the min + product of two matrices with entries in (0, l,..., m } within a factor 1 + l/k using
0(~~(n)g(n22k)~z~(log’2’n
remainder theorem [l], any entry c,,/ of C is in one-to-one correspon e vector ( C I.J
improvement of the well-known upper bound on the bit complexity of the Boolean matrix product (see [l, p. 2431, [5], [6, p. 1541). In the remaining part of the paper, we use this upper bound on arithmetic matrix product and the idea of Romani [8] to derive a good approximation algorithm for the min + product for matrices with nonnecessarily small entries, in the fol!o~jv”liigsense: Let C, D be two matrices with positive integer entries and of the same size. We say that D approximates C within a factor f, f 2 1, if for any pair of the corresponding entries c,.,,d,,,, we have
14 June 1991
od
pp.-d,,/
Therefore,
‘,,Jtmod =
we
)
.
can compute C by computing C, = er hand, for each
P,) C(
%.k mod
P,>( 4.j
mod
PI)
(mod
PI)
k=l
by the definition of C and the laws of modular arithmetic [l]. Let A, = A mod p,, and B, = B mod p,, for 1 < I < q. It follows that Cl = A, X B, for 1 G I G q. Thus, the problem of computing C reduces to the problem of computing the q arithmetic matrix products Cr over the ring Z,,, 1 < I G q, as follows:
1. Find the sequence pl, p2,. . . , pq of primes. 2. Compute the matrices A,, B,, 1 < I < q. 3. For I= 1, . . ., q compute the product C, over
+ log k) log m)
bit operations.
% 4. Compute
C
from
the
residue
matrices
ica-
In order to estimate the bit complexity of arithmetic matrix product of two n x n integer matrices A, B, with entries in (0,. . . , m }, one usually multiplies Ma(n) by Mu(log(nm)). In this way, also an upper bound on the bit complexity of Boolean matrix product by reduction to the arithmetic matrix product over the ring n+, has been derive<: (see [l, p. 2431, [5], [6, p. 1541). e can
p4 be the shortest initial, conLet pi, p2,..., tinuous subsequence of the increasing sequence of prime numbers such that the product of p1 through mn. Next, 1 uct of A and 238
In order to shorten our analysis of the bit complexity of the above method, we define the function g(m) bv g(U) = 2 log U/log’*‘U. Note that g(~)g(“) = G!(u). Hence, we can conclude that the number q of primes in the sequence is 0( g( nm)). Also, by the famous Prime Number Theorem ]4], we have p, = O(log n + log m). Step 1 can be done using the sieve of Eratosthenes for numbers 1 through O(log n + log m ). It takes 0( pi*’ ) divisions of numbers of size O(lo% n + log m), and therefore its bit complexity can be estimated by 0( (log n + log m )1*5Mz4(log’*‘n + log’*‘m)).
iNFORMATION IPROCESSING LETTERS
Volume 38. Number 5
By [4, p. 2911, Step 2, i.e. computing the n2q residua with respect to the CJ primes of size Q(log n + log m), can be done using
14 June 1991
the min + matrix product application, we assume that m can be represented with a polynomial in n number of bits, i.e. r??= 2”O(l).Here, it is sufficient to require
0( n2R4u( g( nm)(log (2)n + log’2’m)) log g(nm)) Ma(n) bit operations. Step 3 takes Ma( n)q arithmetic operations with numbers of size O(log n + log m), and it has A4a( n)g( nm)Mu(log(2)n
+ log’2’m)
bit complexity. Recall that the entries of matrices C,, 1 Q I < q, have size 0( log n + log m ). Therefore, by [ 1, p. 311], the fourth step has
in order to bound the estimate for Step 4 by that for Step 3 asymptotically. Putting everything together, we obtain the following theorem by straightforward calculations. Let A, B be two n x n matrices with integer entries in (0, . . . , m >. Suppose that m = 2’ 1%.?“’ and Ma(n)
0( n2Mu( g( nm)(log “In + logf2’m)) log g( nm )) bit complexity. Note the latter estimate asymptotically bounds from above that on bit complexity of Step 2. By Mu ( n ) = O( n log n logf2’n )
= !2(n2 log n)
= Q( n2 1og”‘n)
or m = 2”!“” and Ma(n)
= !2(n2 log n),
respectively. The arithmetic matrix product of A and B can be computed using 0( Ma(n)g(nm)Mu(log
(see [lo] or [l, pp. 270-2741) and
log n + log log m))
bit operations. g( nm ) = 0( logf2’n + log’2’m), we can transform 0(n2g(nm)Mu(log
it to (“n + log’2)m) log g( nm
)) .
In turn, the above estimate can be asymptotically bounded from above by that on bit complexity of Step 3 if we assume a reasonable lower bound on Ma(n) and a reasonable upper bound on m in terms of n. Having in mind further applications of our method for arithmetic matrix product to computing the Boolean product and the min + matrix product, we respectively distinguish two cases. In the case related to the Boolean matrix product application, we assume that m can be represented with a poly-logarithmic in H number of bits, i.e. m = 20% nP’i* Then, it is enough to assume Ma(n)
= st( n2 logt2’n)
By reducing the Boolean matrix product to an arithmetic one over the ring Z,,, , and using the above theorem for m = n and Ma(n)
= Q(n2 log’2’n),
we can improve the upper bound on bit complexity of the Boolean matrix product given in [ 1, p. 2431 by an Q ( log(2)n/log(4)n ) factor (see also [6, p. 1541).
Ma(n)
= Stjn’ logt2)n),
the Boolean marrix product of two n X n matrices takes O( Ma ( n ) log n log’3’r? log’4)n)
in order to bound the estimate for Step 4 by that for Step 3 asymptotically. In the case related to
bit operations. 239
By Theorem 5.7 in [l], we obtain following corollary.
also the
Corollary 2.3. Assuming Ma(n)
= a( nz log’2’n),
the transitive closure of a Boolean n X n matrix can be computed using 0( Ma ( n ) log n log’3’n logt4’n)
3. 5it complexity of min + matrix product In [B], Romani presented the following algorithm for computing the min + product C of two n X n integer matrices A, B by reduction to an arithmetic matrix product. Algorithm 2.
Set m’ IC:1 + /log n]. Transform A and B to matrices A’ and B’ where a,:, = 2-“‘“~4, and b,;, = 2-m% for 1
log c;,,
roof (sketch). To estimate the bit complexity of the min + product, we follow Mehlhorn’s analysis of Romani’s algorithm [6], performing the substitutions mentioned in the above. By [6], the first step takes 0( n2m log n ) bit operations, and the third one
J For the proof of correctness of Romani’s algorithm see [6] or [B]. In [6], Mehlhorn estimated the bit complexity of Romani’s method of computing the rnin + matrix product. For tutorial reasons, he chose Strassen’s O(n@:) bound for arithmetic matrix multiplication, multiplying it by a naive quadratic bound for integer multiplication to derive the estimate. If we use instead the upper bound given in Theorem 2.1 and the term Mu(n), we will obtain the following theorem. The problem 0:’ computing the min + product of two n x n matrices with integer entries in (0, l,..., m) takes Q( Ma(n)g(n22m)Mu(log’2’n
In order to use the method of Theorem 2.1 to implement the second step of Romani’s algorithm, we rescale the entries of A’ and B’ by multiplying them with 2”‘“. Then, we compute the arithmetic product of the so transformed A’ and B’ using the method of Theorem 2.1, and divide the entries of the resulting product back by 22m’m. The additional multiplications and divisions can be realized by simple shifts, and take O(n’m log n) time. The most costly in our implementation of the second step is computing the arithmetic matrix product which takes 0( Ma(n)g(n
X n2”)Mu(log’2’n
bit operations
by Theorem 2.1.
4. Approximating
1
for 1
240
bit operations provided that Ma( n ) = Q( n ‘log n ) and m = no(‘).
0( n2Mu(log m + logt2’n)).
bit operations.
i
14June 1991
INFORMATIONPROCESSINGLETTERS
Volume38, Number 5
+ log m))
+ log m)) q
the min + lnatrix product
Our algorithm for approximating the min + product relies on the efficient method of computing the min + product for matrices with small integer entries estimated in Theorem 3:l. Algorithm 3.
Input:
n X n matrices A, B with positive integer entries, a positive integer approximation parameter k; Output: an n x n matrix D that approximates the min + product of A and B within a factor 1 + l/k; 1 Initialize a Boolean n x n matrix
E by setting all its entries to fake (setting an entry e,,, to true will mean that the value of c,.~ is determined);
Volume 38, Number 5
INFORMATION
PROCESSING
2. Set m to the maximum value of an entry in A or B, and set d to mJ4k. 3. while m > 2k do begin m”+m+2d;
Transform A and B to matrices A’ and B’ by replacing each entry a,,i in A, and b,., in B which is smaller than m’ with [a,,.,/dj and 1b,.,/d 1, respectively, and replacing each entry greater than m’ with [m/d]; Compute the min + product C’ of A’ and B’; for all entries c,:, do if 7e, , and d X c,:, b m/2 be&n d ‘.J + d x ci’.,; e ‘.J + true; end
LETTERS
14 June 1991
while block that d,.! >, mJ2’. bq.J G m + 4d. Otherwise, 2d 1a,,,/M
] + 2d 1bq, ,/2d
’
d,.J=d[a,,,/d]
+d[bq,J/d(-
a
This combined with + d X b,+,/d
yields c,,J < di .j + 2d - 2. On the other hand, we have
m + m/2; d + d/2; end 4. Transform
A and B to matrices A’ and B’ by replacing each entry a,,/ in A, and b,,, in B which is greater than m + d with m + d; Compute the min + product C’ of A’ and B’; for all entries c,:~ do if ,e,, J then d, .J + c,‘J
In the following lemma, we prove rectness of Algorithm 3.
1 > m,/2’-
and d,,/ is determined in an earlier iteration. We conclude that neither a, cl nor bq,J is greater than m’= m + 2d. In this way we obtain
a,,, + b,.j d d X a,,,/d then
Wz have aiso a, 4 +
which implies di J < Ci.J + 2d - 2. We can estimate 2d - 2 by 2m,/i4k2’-‘) which is not greater than m/2 k. Moreover, we have also di. j >, m/2. It follows that 2d - 2 < di.J/k, and in result 1c,., d,,, 1 < d,, //k. Finally, if d,, J is determined in Step 4, d,. J = c,:, and consequently d,_, = a;,q + 6’ . Note that neither a,‘q nor bi,J is greater than 6’; d since otherwise d,,, was determined in Step 3. Thus, a& = a,.4 and bi, J = bq, J which implies d,.J
the cor-
Lemma 4.1. The matrix
D ap_proximates the min -iproduct of the matrices A and B within a factor 1 + l/k.
=
‘,.J’
q
In the next lemma, we analyse the bit complexity of Algorithm 3. mma 4.2. Assume that k = no(‘) and
Ma(n)
= Q(n2 log N).
Algorithm 3 can be implemented using roof. Let 1 < i < n and 1
0( Ma( n)g( n22k) Mu(log”‘n
+ log k) log m)
bit operations.
,.,=min(a,,,+b,,,ll
Let C,,j = a,., + b,,J, and c,:~ = a,‘q + bi,J. Suppose first that d,,, is determined in Step 3. Let I be the number of the iteration of the while block in which d, / is determined, and let m, stand for the maximum value of an entry in A or B. By the definition of the while block, we have m = m,/2’-’ and d = m,/(4k2’-‘). Ii follows by the test in the
oaf. Step 1,2 can be easily implemented using O(n’log m) bit operations. Let us estimate bit
complexity of the block under the while statement. Note that to compute a,, ,/d and b,_J/d it suffices to chop off the lost 11~; m/4k] bits from the binary representation of a,, J and 6,. /, respectively. Therefore, computing d,m’, transforming A and B into A’ and B’, given C’ partially filling D, all 241
INFORMATION
Volume 38. Number 5
PROCESSING
this can be done performing 0( n’log m ) bit operations. The most costly here is computing C’. Note that the entries in A’ and B’ are of size 0( m ’ ) = 0( k ) (also in Step 4). It follows from Theorem 3.1 that computing C’ takes o( Ma( n)g( n22”) Mu(log’%
to derive a large part of our results in the model of log-space uniform have polylog depth and polyno words, in terms of NC algorithms. Of course, this would require more precise definition of Ma(n) u( n ) in terms of uniform circuits. and
+ log ic ))
steps. Putting everything together and observing that the block under the while statement is iterated O(log m) times, we obtain the total estimate on bit complexity of Algorithm 3. 0 Combining Lemma 4.1 with Lemma 4.2, we obtain the following theorem. Suppose that k = no(‘) and Ma(n) = C!2(n’log n ). The min + product of two n X n matrices with entries in (0, i q. . . 9 m >, can be approximated within a factor 1 + l/k using 0( Ma(n)g(n22”)Mu(log
log n + log k) log m)
I would like to express my appreciation to Rolf Karlsson and an unknown referee for valuable comments, and to Ian Munro for informing me about the state of the art in the min + matrix product problem.
eferences
111A.V.
bit operations.
VI
Aho, J.E. Hopcroft and J.D. Ullman, The Design and Analysis of Cornpurer Algorithms (Addison-Wesley, Reading, MA, 1974). D. Coppersmith, S. Winograd, Matrix multiplication via arithmetic progressions, in: Proc. 19th Ann. ACM Symp. on Theory of Computing (1987) l-6.
[31 W. Dobosiewicz,
Iterating our approximation heuristic for the n-tin + matrix product unfortunately does not yield an efficient approximation method for the matrix closure in the sense of the n-tin + operations. Simply, if the matrix entries are of size Q(n), then k has to be set to 2’(“) in order to achieve a constant approximation factor. This increases the bit complexity of the heuristic to at least cubic which makes the approximation solution unattractive compared with the exact solutions to the closure problem having about the same bit complexity [1,3]. The best, presently-known upper bounds on Ma( n ) and Mu( n ) are achieved by algorithms that can be implemented by uniform aritkmetic circuits which have depth O(log2n) [2] or uniform Boolean circuits which have depth O(log n) [9,10], respectively. Therefore, it would also be possible
242
14 June 199i
LETTERS
plus multiplication.
A more efficient algorithm for the minInternat. J. Comput. Math. 32 (1990)
49-60.
141U. Dudley, iSI
Elementary Number Theory (Freeman, New York, 1970). M.J. Fischer and A.R. Meyer, Boolean matrix multiplication and transitive closure, in: IEEE 12th Ann. Symp. on Switching
and Automata Theory (1971) 129-131. Data Structures and AIgorithms 2: Graph Algorithms and NP-Completeness (Springer, Berlin, 1984)
161K. Mehlhom, 155-158.
171I. Munro, Efficient determination
of the transitive closure of a directed graph, Inform. Process. Left. 1 (1971) 56-58.
PI F. Romani, matrix
Shortest-path problem is not harder than multiplication, Inform. Process. Left. 11 (1980)
134-136.
Pi J&i. Reif and S.R. Tate, Optimal
size integer
division
circuits, in: Proc. 2lst Ann. ACM Symp. on Theory of Compt.ting (1989) 264-273. PO1 A. Schijnhage and V. Strassen, Schnelle Multiplikation Gross Zahlen, Comprdng 7 (1971) 281-292.
1111Y. Pan, How to Multiply Matrices Faster, Lecture Notes in Computer
Science
179 (Springer, Berlin).