- Email: [email protected]
The communication complexity for decentralized evaluation of functions
7 August 1990
Information Processing Letters 35 (1990) 177-182 North-Holland
THF COMMUNICATION OF FUNCTIONS
COMPLEXITY
FOR DECENTRALIZED
EVALUATION
Shyan-Ming YUAN Advanced Technology R.O.C. 31015
Center,
Electronics
Research
Service
Organization,
Industrial
Technology
Research
Institute,
Taiwan,
Communicated by K. Ikeda Received 11 December 1989 Revised 6 April 1990
Decentralized algorithms can be characterized by successive rounds of message interchanges. In each round, all nodes exchange information and carry out computations. The computations at each node in round i are based on the information available at that node after the communication phase of round i. Therefore, the complexity of the communication requirement is a major determining factor of the performance of a decentralized algorithm. In this paper, we study the communication complexity for decentralized evaluation of functions. We show that for any decentralized algorithm of evaluating a function the number of messages required is at least kN([N 1’k] -1) , where N is the number of nodes in the system and k is the number of rounds of message interchanges. In addition, we classify the functions into three types including fully reducible, partially reducible, and irreducible. We than show that for fully reducible functions the number of bits required for decentralized evaluation is at least kN([ N’/k] - l)( B + H), where B is the number of bits required for representing the initial value of each node and H is the number of bits required for the header and tailer of each message. For irreducible functions, the number of bits required for decentralized evaluation is at least N( N - 1) B + kN(I N’lk ] - 1) H.
Keywords:
Decentralized
algorithms, message complexity, bit complexity
1. Introduction Distributed systems can be considered to be a collection of processing nodes connected via a communication network such that processing nodes can interact with each other to serve the processing requirements of the users. In this paper, we assume that the network provides a reliable point-to-point connection between any pair of nodes and that the nodes do not share any common memory and global clock. There are two kinds of control schemes used for designing distributed algorithms. One is centralized control and the other one is decentralized control. A centralized distributed algorithm uses a central coordinator to control the actions of other nodes. Two phase commit protocol [5] is a typical example of centralized algorithms. Decentralized algorithms do not use a central coordinator. All nodes are considered to be identical. In every round, all nodes execute the same program, send out the same number of messages, and receive the same number of messages. In this paper, we study the problem of decentralized evaluating a function with arguments distributed over the whole system (each node possesses one of the arguments initially) and the result of the computation known to all nodes. The total number of messages passed among all processing nodes, called message complexity, and the total number of bits used for all messages exchanged, called bit complexity, are 0020-0190/90/%03.50
0 1990 - Elsevier Science Publishers B.V. (North-Holland)
177
Volume 35, Number 4
INFORMATION
PROCESSING
LE’ITERS
7 August, 1990
measures used for describing the communication complexity 1 of the decentralized algorithms. Decentralized extrema finding [1,6,3,2,11,4] of values distributed over a distributed system and distributed transaction commitment [12,13,10,8,9,14] over a distributed database system are two good examples of decentralized evaluating functions. The simplest way to evaluate a function in a distributed system is by using centralized coordination. In the first phase, the coordinator receives initial values from the rest of nodes and computes the function. In the second phase, the coordinator sends the result to the rest of nodes. It only requires 2( N - 1) messages interchanged. The decentralized extension of this centralized algorithm is to let each node send its initial value to all other nodes. In this way, every node will know all N arguments after one round of message exchange and the relevant computation then can be performed by each node individually. It requires N( N - 1) messages to be passed among nodes in one round of message exchange. Although the message complexity is increased from O(N) to 0( N*) when executing a centralized algorithm in the decentralized manner, some efficient decentralized algorithms have been designed. In [8,9], Lakshman and Agrawala used the finite projective plane as the basis to construct a decentralized consensus protocol with O(Nm) messages in two rounds of message exchanges. Later, Yuan and Agrawala [14] presented a decentralized commit protocol with message complexity of 0( kNN’lk) in k rounds of message exchanges. Independently, Hsieh [7] proposed a similar algorithm for decentralized evaluating the associative and commutative functions with kN( N ‘jk - 1) messages in k rounds of message exchanges. In Section 2, we establish the lower bound of message complexity for decentralized evaluation of functions. We also note that for some of the problems including committing a distributed transaction, finding the extrema, and evaluating the associative and commutative functions, after each round of message exchanges, every node can perform some local computations to aggregate all received values into a constant size message, as in [8,9,14,7], which can capture all useful information of all received messages and be a representative of them. Such functions are called fully reducible functions. Because of this reducible property, the message sizes in all rounds of message interchanges are identical. Thus, the bit complexity of evaluating such functions is equal to the product of the message complexity and the message size. Since the message size is constant, for comparing the algorithms for decentralized evaluating reducible functions, the message complexity is as good as the bit complexity. However, not every function has this property. In general, the message size of a round-( i + 1) message is greater than the message size of a round-i message when no computation can be performed between the message exchanges of round i and (i + 1). The counterparts of the fully reducible functions are called irreducible functions. For evaluating such functions, it is impossible to perform any computation between any two successive rounds of message exchanges to maintain a constant message size. In other words, the computation can only be carried out after a node has the initial values of all nodes. Functions which are not fully reducible nor irreducible are called partially reducible. For this type of functions, after each round of message exchange, it may or may not be able to perform some computations, so the message size cannot be constant as well. For evaluating irreducible or partially reducible functions, the message complexity is no longer a good performance index because it does not consider the effect of variant messages sizes. The bit complexity, which includes the effect of the message size as well as the number of messages, becomes a better choice for comparing decentralized algorithms of evaluating irreducible and partially reducible functions. The lower bound of the bit complexity for decentralized evaluating the irreducible functions is established in Section 3. The concluding remarks are in Section 4. ’ Besides the message and/or bit complexity, the communication cost of a decentralized algorithm is affected by the processing time of handling messages, transmission delay, propagation delay, and so on. Because these factors depend on the actual hardware devices, they are not considered here. 178
Volume
35, Number
INFORMATION
4
PROCESSING
LE’lTERS
7 August,
1990
2. Lower bound of the message complexity The following theorems describe the lower bound of message complexity for decentralized evaluation of any function in an N-node system. Lemma 2.1. Given an integer N, for any k integers p,, pZ, . . _, pk, each of them greater or equal to 1, with nF=‘(p, + 1) > N, CfZ’pi is at least k(lN1/k] - 1). Proof. We claim that the minimal value of C:= 1p, can be obtained by assigning m p, ‘s as (] N ‘lk] - 1) and the other (k - m) p,‘s as (]N1jk] - l), where 0 < m < k and [Ni/k]m[Ni/kJk-m~
N,
~Nl/k]m-l~Nl/k~k-m+l,
Therefore, the minimum of XFZl p, is m(]N’lk] - 1) + (k - m)([N’/“] - 1). This result can be shown by construction. Initially, let every p, be 1. If 2k >, N, then we are done. Otherwise, increasing CF=, p, by 1 in such a way that n,“_,( p, + 1) is maximized. It is easy to see that increasing the smallest p, by 1 will do it. Continuing this procedure until nf=i=,
- 1) + (k - m)([N”k]
Cf=i p, is at least k(lN1/k]
- 1).
- 1) > k([N1/k]
- l),
(1)
q
Theorem 2.2. Any algorithm of decentralized evaluating a function in an N-node system with k rounds of message interchanges requires at least kN([ N’jk] - 1) messages passed among the N nodes. Proof. Assume that for i, 1 G i < k, in the ith round of message interchange, each node sends p, messages So, after round 1, there are at most (p, + 1) nodes having the knowledge about the initial value of node 1. After round 2, there are at most ( p’ + l)( p2 + 1) nodes having the knowledge about the initial value of node 1. Therefore, after round k, there are at most nF= i( p, + 1) nodes having the knowledge about the initial value of node 1. Since only k rounds of message interchanges are used, nF= i( p, + 1) should be at least N. Otherwise, some nodes would have no knowledge about the initial value of the node 1, so they will not be able to compute the final result. Since each node sends out p, messages in round i, 1 < i < k, the total number of messages generated is N Cf=i p,. Because n,“_,( p, + 1) > N and for any set of k integers, p,, pz, . . . , pk, each of them greater or equal to 1, If+ p, is at least k(lN1/k] - 1) (Lemma 2.1) the total number of messages N Xf=i pi is at least kN([N1/k] - 1). q out.
Yuan [14] and Hsieh [7] have presented decentralized algorithms for transaction commitments and evaluating associative and commutative functions with message complexity of kN( N1lk - l), which is the same as the lower bound obtained here when N1lk 1s an integer. Therefore, this is the absolute lower bound of the message complexity for decentralized evaluating any kind of functions by using k rounds of message interchanges in an N-node system.
3. Lower bound of the bit complexity For fully reducible functions, the message size of all rounds is constant, so the lower bound of the bit complexity for decentralized evaluating a fully reducible function is equal to the product of the lower 179
Volume
35, Number
INFORMATION
4
PROCESSING
LETTERS
I August,
1990
bound of the message complexity and the constant message size. Therefore, the lower bound of the bit complexity for decentralized evaluation of fully reducible functions is kN([N’/kj - l)( B -t H), where B is the number of bits for representing the initial value of each node and H is the number of bits required for the header and tailer of each message. In evaluating the irreducible functions, the computation phase of round i cannot yield a reduction in the number of bits needed for transferring all collected values so far. For this type of computation, if the information contained by a round-i message is 4 bits and every node receives pi messages in round i, then the information contained by a round-(i + 1) message should be greater than 4 bits because no information aggregation can be done. The following theorem establishes the lower bound of the bit complexity for decentralized evaluating irreducible functions. Theorem 3.1. For decentralized evaluating irreducible functions in an N-node system, if k rounds of message interchanges are used, each initial value requires B bits, and each message requires H bits of overhead, then the total number of bits required for all messages is at least N( N - 1) B + kN(l N1/k] - 1) H. Proof. Assume that Vi, 1 < i < k, in the ith round of message interchange, each site sends p, messages out and receives pi messages. First we show that each message sent in round i contains initial values of at least N/nt=,( pJ + 1) nodes, where 2 < i < k. Base case. After round k, each node should contain initial values of all N nodes. Because each node receives pk messages in round k, the messages sent in round k should contain initial values of at least N/( pk + 1) nodes. Otherwise no node can have all initial values to perform the computation. Induction case. Assume that each message sent in round i + 1 contains initial values of at least N/n:= ;+i( p, + 1) nodes. Since each node receives p, messages in round i, the messages sent in round i should contain initial values of at least N/(( p, + l)r’I,“,,+ i( p, + 1)) = N/n,k=i( p, + 1) nodes. Since every message sent in round i, 2 < i < k, contains initial values of at least N/n,k= ;( p, + 1) nodes and k
n j=l
(Pj+l)
>N>
k
k
(2) the total number of bits (TB) required for all messages is as follows: k
TB=Nz
(pi.mj), i=l
where m, is the size of messages sent in round i
1)
180
Volume
35, Number
4
INFORMATION
i
‘I+
n,k,,(p,+l)
rI(P,+1)-P1-1
1990
P,
+NHc
j-1
ny=2( Pj + ‘>
I August,
k
i=l k
N
N+
LETTERS
k
N =NB
PROCESSING
+NHc
P, i=l
PJ)=*(P,fl)-N rq=z(P,+l)
+NH;P, i=l
>NB(N-l)+NH;pi.
(3) i=l
From Lemma 2.1, EF=, pi >, k([ N”“ ] - l), the lower bound of bit complexity for decentralized evaluating •I irreducible functions is N( N - 1)B + kN([ N’lk] - l)H. By applying algorithms proposed in [14] or [7] to evaluate irreducible functions, it can be easily shown that the bit complexity is N( N - l)B + kN( N ‘/k - l)H. Therefore, the bit complexity established here is the absolute lower bound of decentralized evaluating irreducible functions with k rounds of message exchanges in an N-node system.
4. Conclusion
We have shown that the lower bound of message complexity of decentralized evaluating any function is kN([ N1jk] - 1) in an N-node system with k rounds of message exchanges. We also classified the functions into three classes: fully reducible, partially reducible, and irreducible. For evaluating the reducible functions, because the message size is constant for all rounds, the lower bound of the bit complexity is kN([ N1jk] - l)( B + H). For evaluating the irreducible functions, because the message size is increased after each round, the lower bound of the bit complexity is shown to be N( N - l)B + kN(lN”k] - l)H. For evaluating the partially reducible functions, although the message size is not constant, it is not increased as much as the irreducible functions. Therefore, the lower bound of the bit complexity for decentralized evaluating the partially reducible functions must be between the lower bounds of the fully reducible and the irreducible functions. Since there are existing algorithms [14,7] which can achieve those lower bounds, we know that these lower bounds are the absolute lower bounds of decentralized evaluating functions.
Acknowledgment
We are grateful to the referees for their suggestions toward improving the paper. 181
Volume
35, Number
4
INFORMATION
PROCESSING
LETTERS
7 August,
1990
References [l] E. Chang and R. Roberts, An improved algorithm for decentralized extrema-finding in circular configurations of processors, Comm. ACM 22 (1979) 281-283. (21 D. Dolev, M. Klawe and M. Rodeh, An o(n log n) unidirectional distributed algorithm for extrema finding in a circle, J. Algorithms 3 (1982) 245-260. [3] W.R. Franklin, On an improved algorithm for decentralconfigurations of ized extrema-finding in circular processors, Comm. ACM 25 (1982) 336-337. (41 G.N. Frederickson and N. Lynch, The impact of synchronous communication on the problem of electing a leader in a ring, in: Proc. 16th ACM Symposium on Theory of Computing, Washington, DC (ACM, New York, 1984) 493-503. [5] J.N. Gray, Notes on database operating systems, in: Operating Systems: An Advanced Course (Springer, Berlin, 1979). [6] D. Hirschberg and J. Sinclair, Decentralized extrema-finding in circular configurations of processors, Comm. ACM 23 (1980) 627-628. [7] C.S. Hsieh, Decentralized evaluation of associative and commutative functions, in: Proc. 9th International Conference on Distributed Computing Systems, Newport Beach, CA (IEEE Press, New York, 1989) 9-11. [8] T.V. Lakshman and A.K. Agrawala, o(n&) decentralized
182
[9]
[lo]
[ll]
[12]
[13]
[14]
commit protocols, in: Proc. 5th International Symposium on Reliability in Distributed Software and Database Systems, Los Angeles, CA (IEEE Press, New York, 1986). T.V. Lakshman and A.K. Agrawala, Efficient decentralized consensus protocols, IEEE Trans. Software Engrg. 12 (5) (1986) 600-607. C. Mohan and B. Lindsay, Efficient commit protocols for the tree of processes model of distributed transactions, in: Proc. 2nd ACM SIGACT/ SIGOPS Symposium on Principles of Distributed Computing, Montreal, Canada (1983) 76-80. G.L. Peterson, An o(n log n) unidirectional algorithm for the circular extrema problem, ACM Trans. Compuf. Systems 4 (1982) 758-762. D. Skeen, Nonblocking commit protocols, in: ACM SIGMOD International Conference on Management of Data (ACM, New York, 1981) 133-142. D. Skeen and M. Stonebraker, A formal model of crash recovery in a distributed system, IEEE Trans. Software Engrg. 9 (1983) 219-228. S. Yuan and A.K. Agrawala, A class of optimal decentralized commit protocols, in: Proc. 8th International Conference on Distributed Computing Systems, San Jose, CA (IEEE Press, New York, 1988) 234-241.
Information Processing Letters 35 (1990) 177-182 North-Holland
THF COMMUNICATION OF FUNCTIONS
COMPLEXITY
FOR DECENTRALIZED
EVALUATION
Shyan-Ming YUAN Advanced Technology R.O.C. 31015
Center,
Electronics
Research
Service
Organization,
Industrial
Technology
Research
Institute,
Taiwan,
Communicated by K. Ikeda Received 11 December 1989 Revised 6 April 1990
Decentralized algorithms can be characterized by successive rounds of message interchanges. In each round, all nodes exchange information and carry out computations. The computations at each node in round i are based on the information available at that node after the communication phase of round i. Therefore, the complexity of the communication requirement is a major determining factor of the performance of a decentralized algorithm. In this paper, we study the communication complexity for decentralized evaluation of functions. We show that for any decentralized algorithm of evaluating a function the number of messages required is at least kN([N 1’k] -1) , where N is the number of nodes in the system and k is the number of rounds of message interchanges. In addition, we classify the functions into three types including fully reducible, partially reducible, and irreducible. We than show that for fully reducible functions the number of bits required for decentralized evaluation is at least kN([ N’/k] - l)( B + H), where B is the number of bits required for representing the initial value of each node and H is the number of bits required for the header and tailer of each message. For irreducible functions, the number of bits required for decentralized evaluation is at least N( N - 1) B + kN(I N’lk ] - 1) H.
Keywords:
Decentralized
algorithms, message complexity, bit complexity
1. Introduction Distributed systems can be considered to be a collection of processing nodes connected via a communication network such that processing nodes can interact with each other to serve the processing requirements of the users. In this paper, we assume that the network provides a reliable point-to-point connection between any pair of nodes and that the nodes do not share any common memory and global clock. There are two kinds of control schemes used for designing distributed algorithms. One is centralized control and the other one is decentralized control. A centralized distributed algorithm uses a central coordinator to control the actions of other nodes. Two phase commit protocol [5] is a typical example of centralized algorithms. Decentralized algorithms do not use a central coordinator. All nodes are considered to be identical. In every round, all nodes execute the same program, send out the same number of messages, and receive the same number of messages. In this paper, we study the problem of decentralized evaluating a function with arguments distributed over the whole system (each node possesses one of the arguments initially) and the result of the computation known to all nodes. The total number of messages passed among all processing nodes, called message complexity, and the total number of bits used for all messages exchanged, called bit complexity, are 0020-0190/90/%03.50
0 1990 - Elsevier Science Publishers B.V. (North-Holland)
177
Volume 35, Number 4
INFORMATION
PROCESSING
LE’ITERS
7 August, 1990
measures used for describing the communication complexity 1 of the decentralized algorithms. Decentralized extrema finding [1,6,3,2,11,4] of values distributed over a distributed system and distributed transaction commitment [12,13,10,8,9,14] over a distributed database system are two good examples of decentralized evaluating functions. The simplest way to evaluate a function in a distributed system is by using centralized coordination. In the first phase, the coordinator receives initial values from the rest of nodes and computes the function. In the second phase, the coordinator sends the result to the rest of nodes. It only requires 2( N - 1) messages interchanged. The decentralized extension of this centralized algorithm is to let each node send its initial value to all other nodes. In this way, every node will know all N arguments after one round of message exchange and the relevant computation then can be performed by each node individually. It requires N( N - 1) messages to be passed among nodes in one round of message exchange. Although the message complexity is increased from O(N) to 0( N*) when executing a centralized algorithm in the decentralized manner, some efficient decentralized algorithms have been designed. In [8,9], Lakshman and Agrawala used the finite projective plane as the basis to construct a decentralized consensus protocol with O(Nm) messages in two rounds of message exchanges. Later, Yuan and Agrawala [14] presented a decentralized commit protocol with message complexity of 0( kNN’lk) in k rounds of message exchanges. Independently, Hsieh [7] proposed a similar algorithm for decentralized evaluating the associative and commutative functions with kN( N ‘jk - 1) messages in k rounds of message exchanges. In Section 2, we establish the lower bound of message complexity for decentralized evaluation of functions. We also note that for some of the problems including committing a distributed transaction, finding the extrema, and evaluating the associative and commutative functions, after each round of message exchanges, every node can perform some local computations to aggregate all received values into a constant size message, as in [8,9,14,7], which can capture all useful information of all received messages and be a representative of them. Such functions are called fully reducible functions. Because of this reducible property, the message sizes in all rounds of message interchanges are identical. Thus, the bit complexity of evaluating such functions is equal to the product of the message complexity and the message size. Since the message size is constant, for comparing the algorithms for decentralized evaluating reducible functions, the message complexity is as good as the bit complexity. However, not every function has this property. In general, the message size of a round-( i + 1) message is greater than the message size of a round-i message when no computation can be performed between the message exchanges of round i and (i + 1). The counterparts of the fully reducible functions are called irreducible functions. For evaluating such functions, it is impossible to perform any computation between any two successive rounds of message exchanges to maintain a constant message size. In other words, the computation can only be carried out after a node has the initial values of all nodes. Functions which are not fully reducible nor irreducible are called partially reducible. For this type of functions, after each round of message exchange, it may or may not be able to perform some computations, so the message size cannot be constant as well. For evaluating irreducible or partially reducible functions, the message complexity is no longer a good performance index because it does not consider the effect of variant messages sizes. The bit complexity, which includes the effect of the message size as well as the number of messages, becomes a better choice for comparing decentralized algorithms of evaluating irreducible and partially reducible functions. The lower bound of the bit complexity for decentralized evaluating the irreducible functions is established in Section 3. The concluding remarks are in Section 4. ’ Besides the message and/or bit complexity, the communication cost of a decentralized algorithm is affected by the processing time of handling messages, transmission delay, propagation delay, and so on. Because these factors depend on the actual hardware devices, they are not considered here. 178
Volume
35, Number
INFORMATION
4
PROCESSING
LE’lTERS
7 August,
1990
2. Lower bound of the message complexity The following theorems describe the lower bound of message complexity for decentralized evaluation of any function in an N-node system. Lemma 2.1. Given an integer N, for any k integers p,, pZ, . . _, pk, each of them greater or equal to 1, with nF=‘(p, + 1) > N, CfZ’pi is at least k(lN1/k] - 1). Proof. We claim that the minimal value of C:= 1p, can be obtained by assigning m p, ‘s as (] N ‘lk] - 1) and the other (k - m) p,‘s as (]N1jk] - l), where 0 < m < k and [Ni/k]m[Ni/kJk-m~
N,
~Nl/k]m-l~Nl/k~k-m+l,
Therefore, the minimum of XFZl p, is m(]N’lk] - 1) + (k - m)([N’/“] - 1). This result can be shown by construction. Initially, let every p, be 1. If 2k >, N, then we are done. Otherwise, increasing CF=, p, by 1 in such a way that n,“_,( p, + 1) is maximized. It is easy to see that increasing the smallest p, by 1 will do it. Continuing this procedure until nf=i=,
- 1) + (k - m)([N”k]
Cf=i p, is at least k(lN1/k]
- 1).
- 1) > k([N1/k]
- l),
(1)
q
Theorem 2.2. Any algorithm of decentralized evaluating a function in an N-node system with k rounds of message interchanges requires at least kN([ N’jk] - 1) messages passed among the N nodes. Proof. Assume that for i, 1 G i < k, in the ith round of message interchange, each node sends p, messages So, after round 1, there are at most (p, + 1) nodes having the knowledge about the initial value of node 1. After round 2, there are at most ( p’ + l)( p2 + 1) nodes having the knowledge about the initial value of node 1. Therefore, after round k, there are at most nF= i( p, + 1) nodes having the knowledge about the initial value of node 1. Since only k rounds of message interchanges are used, nF= i( p, + 1) should be at least N. Otherwise, some nodes would have no knowledge about the initial value of the node 1, so they will not be able to compute the final result. Since each node sends out p, messages in round i, 1 < i < k, the total number of messages generated is N Cf=i p,. Because n,“_,( p, + 1) > N and for any set of k integers, p,, pz, . . . , pk, each of them greater or equal to 1, If+ p, is at least k(lN1/k] - 1) (Lemma 2.1) the total number of messages N Xf=i pi is at least kN([N1/k] - 1). q out.
Yuan [14] and Hsieh [7] have presented decentralized algorithms for transaction commitments and evaluating associative and commutative functions with message complexity of kN( N1lk - l), which is the same as the lower bound obtained here when N1lk 1s an integer. Therefore, this is the absolute lower bound of the message complexity for decentralized evaluating any kind of functions by using k rounds of message interchanges in an N-node system.
3. Lower bound of the bit complexity For fully reducible functions, the message size of all rounds is constant, so the lower bound of the bit complexity for decentralized evaluating a fully reducible function is equal to the product of the lower 179
Volume
35, Number
INFORMATION
4
PROCESSING
LETTERS
I August,
1990
bound of the message complexity and the constant message size. Therefore, the lower bound of the bit complexity for decentralized evaluation of fully reducible functions is kN([N’/kj - l)( B -t H), where B is the number of bits for representing the initial value of each node and H is the number of bits required for the header and tailer of each message. In evaluating the irreducible functions, the computation phase of round i cannot yield a reduction in the number of bits needed for transferring all collected values so far. For this type of computation, if the information contained by a round-i message is 4 bits and every node receives pi messages in round i, then the information contained by a round-(i + 1) message should be greater than 4 bits because no information aggregation can be done. The following theorem establishes the lower bound of the bit complexity for decentralized evaluating irreducible functions. Theorem 3.1. For decentralized evaluating irreducible functions in an N-node system, if k rounds of message interchanges are used, each initial value requires B bits, and each message requires H bits of overhead, then the total number of bits required for all messages is at least N( N - 1) B + kN(l N1/k] - 1) H. Proof. Assume that Vi, 1 < i < k, in the ith round of message interchange, each site sends p, messages out and receives pi messages. First we show that each message sent in round i contains initial values of at least N/nt=,( pJ + 1) nodes, where 2 < i < k. Base case. After round k, each node should contain initial values of all N nodes. Because each node receives pk messages in round k, the messages sent in round k should contain initial values of at least N/( pk + 1) nodes. Otherwise no node can have all initial values to perform the computation. Induction case. Assume that each message sent in round i + 1 contains initial values of at least N/n:= ;+i( p, + 1) nodes. Since each node receives p, messages in round i, the messages sent in round i should contain initial values of at least N/(( p, + l)r’I,“,,+ i( p, + 1)) = N/n,k=i( p, + 1) nodes. Since every message sent in round i, 2 < i < k, contains initial values of at least N/n,k= ;( p, + 1) nodes and k
n j=l
(Pj+l)
>N>
k
k
(2) the total number of bits (TB) required for all messages is as follows: k
TB=Nz
(pi.mj), i=l
where m, is the size of messages sent in round i
1)
180
Volume
35, Number
4
INFORMATION
i
‘I+
n,k,,(p,+l)
rI(P,+1)-P1-1
1990
P,
+NHc
j-1
ny=2( Pj + ‘>
I August,
k
i=l k
N
N+
LETTERS
k
N =NB
PROCESSING
+NHc
P, i=l
PJ)=*(P,fl)-N rq=z(P,+l)
+NH;P, i=l
>NB(N-l)+NH;pi.
(3) i=l
From Lemma 2.1, EF=, pi >, k([ N”“ ] - l), the lower bound of bit complexity for decentralized evaluating •I irreducible functions is N( N - 1)B + kN([ N’lk] - l)H. By applying algorithms proposed in [14] or [7] to evaluate irreducible functions, it can be easily shown that the bit complexity is N( N - l)B + kN( N ‘/k - l)H. Therefore, the bit complexity established here is the absolute lower bound of decentralized evaluating irreducible functions with k rounds of message exchanges in an N-node system.
4. Conclusion
We have shown that the lower bound of message complexity of decentralized evaluating any function is kN([ N1jk] - 1) in an N-node system with k rounds of message exchanges. We also classified the functions into three classes: fully reducible, partially reducible, and irreducible. For evaluating the reducible functions, because the message size is constant for all rounds, the lower bound of the bit complexity is kN([ N1jk] - l)( B + H). For evaluating the irreducible functions, because the message size is increased after each round, the lower bound of the bit complexity is shown to be N( N - l)B + kN(lN”k] - l)H. For evaluating the partially reducible functions, although the message size is not constant, it is not increased as much as the irreducible functions. Therefore, the lower bound of the bit complexity for decentralized evaluating the partially reducible functions must be between the lower bounds of the fully reducible and the irreducible functions. Since there are existing algorithms [14,7] which can achieve those lower bounds, we know that these lower bounds are the absolute lower bounds of decentralized evaluating functions.
Acknowledgment
We are grateful to the referees for their suggestions toward improving the paper. 181
Volume
35, Number
4
INFORMATION
PROCESSING
LETTERS
7 August,
1990
References [l] E. Chang and R. Roberts, An improved algorithm for decentralized extrema-finding in circular configurations of processors, Comm. ACM 22 (1979) 281-283. (21 D. Dolev, M. Klawe and M. Rodeh, An o(n log n) unidirectional distributed algorithm for extrema finding in a circle, J. Algorithms 3 (1982) 245-260. [3] W.R. Franklin, On an improved algorithm for decentralconfigurations of ized extrema-finding in circular processors, Comm. ACM 25 (1982) 336-337. (41 G.N. Frederickson and N. Lynch, The impact of synchronous communication on the problem of electing a leader in a ring, in: Proc. 16th ACM Symposium on Theory of Computing, Washington, DC (ACM, New York, 1984) 493-503. [5] J.N. Gray, Notes on database operating systems, in: Operating Systems: An Advanced Course (Springer, Berlin, 1979). [6] D. Hirschberg and J. Sinclair, Decentralized extrema-finding in circular configurations of processors, Comm. ACM 23 (1980) 627-628. [7] C.S. Hsieh, Decentralized evaluation of associative and commutative functions, in: Proc. 9th International Conference on Distributed Computing Systems, Newport Beach, CA (IEEE Press, New York, 1989) 9-11. [8] T.V. Lakshman and A.K. Agrawala, o(n&) decentralized
182
[9]
[lo]
[ll]
[12]
[13]
[14]
commit protocols, in: Proc. 5th International Symposium on Reliability in Distributed Software and Database Systems, Los Angeles, CA (IEEE Press, New York, 1986). T.V. Lakshman and A.K. Agrawala, Efficient decentralized consensus protocols, IEEE Trans. Software Engrg. 12 (5) (1986) 600-607. C. Mohan and B. Lindsay, Efficient commit protocols for the tree of processes model of distributed transactions, in: Proc. 2nd ACM SIGACT/ SIGOPS Symposium on Principles of Distributed Computing, Montreal, Canada (1983) 76-80. G.L. Peterson, An o(n log n) unidirectional algorithm for the circular extrema problem, ACM Trans. Compuf. Systems 4 (1982) 758-762. D. Skeen, Nonblocking commit protocols, in: ACM SIGMOD International Conference on Management of Data (ACM, New York, 1981) 133-142. D. Skeen and M. Stonebraker, A formal model of crash recovery in a distributed system, IEEE Trans. Software Engrg. 9 (1983) 219-228. S. Yuan and A.K. Agrawala, A class of optimal decentralized commit protocols, in: Proc. 8th International Conference on Distributed Computing Systems, San Jose, CA (IEEE Press, New York, 1988) 234-241.