An optimized RGA supporting selective undo for collaborative text editing systems

Journal of Parallel and Distributed Computing 132 (2019) 310–330

Contents lists available at ScienceDirect

J. Parallel Distrib. Comput. journal homepage: www.elsevier.com/locate/jpdc

An optimized RGA supporting selective undo for collaborative text editing systems ∗

Xiao Lv a,b , Fazhi He a , , Weiwei Cai a , Yuan Cheng a a b

School of Computer Science, Wuhan University, China College of Computer Engineering, Naval University of Engineering, China

highlights • • • •

An efficient algorithm called ORGAU with integrated support of do and selective undo. It has a time complexity lower than typical CRDT algorithms while keeping the same space complexity. The experiment evaluations show that the proposed algorithm has better computing performance. Both the correctness and the intention preserving of the proposed algorithm have been formally proved.

article

info

Article history: Received 17 January 2017 Received in revised form 5 July 2018 Accepted 14 May 2019 Available online 23 May 2019 Keywords: CSCW (Computer Supported Cooperative Work) Collaborative text editing Selective undo CRDT(Commutative Replicated Data Types) RGA(Replicated Growable Array)

a b s t r a c t Collaboration plays a key role in distributed applications. As a fundamental vehicle for collaboration, collaborative text editing systems have been an important field within CSCW. More recently, with the increasing popularity of cloud computing, collaborative text editing systems move towards large-scale collaborations based on the cloud computing/cloud platform. The computing performance is the key factor of success for large-scale collaborations. CRDT algorithms have been proved to outperform traditional algorithms in publications. However, how to support selective undo has been a challenging issue for existing CRDT algorithms. This paper proposes an efficient CRDT algorithm called ORGAU that provides integrated do and selective undo efficiently. The correctness and operation intentions preserving of the proposed algorithm under an integrated do/undo framework are formally proved. Compared with the typical CRDT algorithms, the proposed algorithm has better computing performance both in theoretical analysis and experimental evaluation while keeping the same space complexity. © 2019 Elsevier Inc. All rights reserved.

1. Introduction With the rapid development of the network and distributed computing systems, more and more principles and applications of distributed systems have been proposed, such as theory and £ practice [22,65], model and framework [21,54], parallel computing [3,5,36,55,76], graphic processing [31], task scheduling and simulation [7,10,11,74] and collaborative interaction [14,15]. As a fundamental vehicle of distributed applications, collaborative text editing systems have been a significant field within CSCW [17,40, 60]. Plenty of collaborative algorithms have been developed to support real-world collaborative work, e.g. Jupiter [39], Nice [53], 2D spreadsheets [62], documents [12,63,73], 2D images [16,83, 84], 3D digital media design [8,58], collaborative game [85], collaborative 3D CAD [18,34]. ∗ Corresponding author. E-mail address: [email protected] (F. He). https://doi.org/10.1016/j.jpdc.2019.05.005 0743-7315/© 2019 Elsevier Inc. All rights reserved.

More recently, with the increasing popularity of cloud computing, collaborative text editing systems move towards largescale collaborations based on the cloud computing/cloud platform [2,19,35,37], which deserve further to deal with new technical challenges. Firstly, a selective undo mechanism is indispensable in a largescale collaboration, because erroneous operations should be allowed to be undone by any collaborator at any time [43,47,56, 57]. However, only few works concentrate on supporting undo operations, such as selective undo for atomic objects [51,61,70] and coarse-grained objects [33,80,81]. Secondly, a large amount of data updates needs to be integrated in the same shared document, which generally causes the decrease of computing performance. Therefore, the computing performance is the key factor of the success for a large-scale collaboration [2]. However, most algorithms take time quadratic or time exponential in the size of the operation history [43,61]. Thirdly, how to formally prove the correctness of do and selective undo is a challenging issue.

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330 Table 1 Undo features supported by OT-based algorithms.

311

Table 2 Undo features supported by CRDT-based undo algorithms.

Work Undo order

Object granularity Time complexity Puzzle

Work

Undo order

Object granularity

Time complexity

[43] [47] [56] [57] [61] [51]

Character Character Character Character Character Character

[70] [80] [81] [33]

Any Any Any Any

Unbreakable line Breakable string Breakable string Breakable string

Logarithmic Linear Linear Best-case:constant worst-case:exponential

Any Chronological Any Any Any Any

Quadratic Linear Exponential Exponential Exponential Linear

Violation of TP2

For example, OT(Operational Transformation) puzzle is one longlasting issue, such as violation of TP1 (Transformation Property 1) for Ellis’s IT (Inclusive Transformation) and violation of TP2 (Transformation Property 2) for Ressel’s IT [46,64]. This paper proposes an efficient CRDT algorithm called ORGAU (Optimized RGA supporting selective undo) that integrates do and selective undo. This work enhances a previous JPDC work called RGA [48]. Its correctness has been formally proved [49]. Unfortunately, the original RGA only supports to integrate do operations. We extend the work to support both do and selective undo. In addition, we discuss how to extend the proposed algorithm to cloud computing/cloud storage environment, which gives the direction of future investigation. This work makes the following contributions. (1) The proposed algorithm supports do and selective undo in collaborative text editing systems. (2) Theoretical analysis and experimental evaluation show that the proposed algorithm outperforms the typical CRDT algorithms. (3) Formal proofs are provided to show the correctness and operation intentions preserving of the proposed algorithm. The rest of this paper is organized as follows. Section 2 gives the background and related work. Section 3 presents the proposed approach in detail. Section 4 provides an integrated do/ undo example. Sections 5 and 6 compare the computing performance of our approach with the typical CRDT algorithms both in theory and experimental evaluation. Section 7 presents an overall scheme of cloud-based collaborative editing service. Section 8 summarizes contributions and future directions. Appendixes give the formal proofs of the correctness and consistency of operation intentions. 2. Background and related work Over the past 25 years, some mainstream concurrency control techniques provide powerful tools for implementing selective undo in collaborative text editing [50], such as OT and CRDT. OT algorithms have been considered as standard optimistic consistency control methods for collaborative text editing [77]. Various OT algorithms have been applied in collaborative applications, e.g. CoWord,1 Google Wave/ Docs,2 IBM OpenCoWeb.3 Most OT researches concentrate on supporting do operations, such as SOCT3 and SOCT4 [67], TIBOT [28], ABT [27]. Only few works focus on supporting selective undo [56,57,61]. Table 1 shows undo features of typical OT-based undo algorithms in detail. As shown in Table 1, the computing performance of OT-based undo algorithms deserves further to be improved. CRDT algorithms have been proposed as a new class of collaborative editing algorithms [6,44,52]. Many references have reported that CRDT algorithms have good scalability and outperform traditional algorithms by orders of magnitudes [1,13,32,68]. However, only few CRDT algorithms support selective undo, such 1 http://www.codoxware.com. 2 http://docs.google.com. 3 https://github.com/opencoweb/coweb#readme.

Puzzle

as supporting selective undo for atomic objects [70] and coarsegrained objects [33,80,81]. Table 2 shows undo features of typical CRDT-based undo algorithms. As shown in Table 2, researches in the literatures [33,80,81] support for coarse-grained objects and not specially for atomic objects. Therefore, string-wise selective undo with high performance for atomic objects cannot be guaranteed. 3. The proposed approach 3.1. The overall do/undo framework The overall do/undo framework is shown in Fig. 1. A collaborative text editing system consists of a group of collaborative sites. Each collaborative site maintains a replicated data structure (Data Structure), the operation history (Log) and the user’s view (User −View ). Data Structure includes a hash table (Hash Table) and a single linked list (List). Hash Table stores all visible and invisible blocks. List links all visible and invisible blocks in a total order. Log stores all executed operations including local operations and remote operations. User −View provides the interaction interface for collaborative users. For example, in Fig. 1, the shared text document is as the interaction interface. Actually, User −View is a single linked list, which stores all the visible blocks of List. Based on the framework in Fig. 1, the whole control procedure is as follows. When a new session starts, collaborative users concurrently generate local insertions and deletions based on User −View . At the same time, collaborative users can generate local undo of any earlier executed operations stored in Log. In local sites, local (do/undo) operations are executed and stored in Log. Then, local operations are transformed to remote operations and broadcast to remote sites. When other collaborative sites receive remote (do/undo) operations, the remote operations are executed and stored in Log. The synchronization between User −View and List is needed to make the effects of integrated updates appear in User −View . 3.2. Related definitions Each operation O has six elements, which is defined as follows. Definition 1. An operation O is a six-tuple ⟨type, l, tar_key, cur_key, con, tar_o⟩ where (1) type is an enum variable, which shows the type of an operation O, O may be an insertion, a deletion, an undo of the executed operation. (2) l is a bool variable whose value is true or false. The value true represents local operations, the value false represents remote operations. (3) tar_key is the identifier of the target operation. (4) cur_key is the identifier of the current operation. (5) con is the content of the inserted block or the deleted block. (6) tar_o is the target operation to be undone, which is specially used for selective undo.

312

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

Fig. 1. The overall do/undo framework.

The element type records the type of an operation, such as an insertion, a deletion and an undo operation. When integrating an operation, different integrating algorithms are invoked according to the type of an operation. The element l is used to judge whether an operation is a local operation or a remote operation. Therefore, the elements type and l together determine which function should further be invoked. For example, when the value of type is Ins and the value of l is true, the algorithm that integrates a local insertion is invoked. The elements tar_key and cur_key adopt the unique identifiers S4v ectors, which have been proposed in RGA [48]. For an insertion, tar_key is the unique identifier of the target object, after which a new object is inserted. For a deletion, tar_key is the unique identifier of the deleted object. For an undo operation, two cases need to be considered. If the target operation to be undone is an insertion, cur_key of an insertion is used to find the inserted object. If the target operation to be undone is a deletion, tar_key of a deletion is used to find the deleted object. The object of the operation is the block. Each block represents the atomic character or unbreakable string, which contains five members, the following gives the definition of a block. Definition 2. A block is a five-tuple where (1) cur_key is the identifier of each block. (2) v isible is an integer variable whose value can be positive, negative or 0. (3) con is the content of the block, which may be the atomic character or unbreakable string. (4) next is a pointer, which points the next block in the single linked list(List). (5) link is a pointer, which is used for the double chaining in the hash table(Hash Table). The element v isible has two uses. Firstly, it is used to judge the visibility of a block. When a new block is inserted in List and Hash Table, the value of block.v isible is initialized to one. When a block is deleted, it is not removed from List and Hash Table, but the value of block.v isible is decreased by one. When multiple concurrent deletions delete the same block, the values of block.v isible are reduced by one with multiple times. That is the reason why the value may be negative. When the value of

v isible is less than or equal to zero, the block is preserved as a tombstone and stored in Hash Table and List, but it does not appear in User −View . When the value of v isible is greater than zero, it is not only stored in Hash Table and List but also appears in User −View . Secondly, in order to support selective undo, it is used to record the effect of selective undo. The following gives the definition and interpretation of effect of selective undo in detail. Definition 3. A block is a tombstone iff : block.v isible <= 0. A block is not a tombstone iff : block.v isible > 0. Definition 4. A block is deleted, block.v isible = block.v isible − 1. A block is inserted, block.v isible = 1. Based on Definitions 3 and 4, when a (local/remote) operation is integrated, block.v isible needs to be calculated. For example, when a (local/remote) insertion is integrated, block.v isible is set to one. When a (local/remote) deletion is integrated, block.v isible is reduced by one. When List and User −View are synchronized, it is necessary to judge the value of block.v isible. Furthermore, the synchronization between List and User −View needs to make all visible blocks appear in User −View . As a result, all blocks of List need to be traversed one by one. When block.v isible is greater than zero, the block is placed and appeared in User −View . Definition 5. The effect of selective undo (1) When a deletion is undone, block.v isible = block.v isible + 1. (2) When an insertion is undone, block.v isible = block.v isible − 1. Anyway, different interpretations of selective undo result in different definitions, different definitions lead to different semantics of selective undo, different semantics generate different effects of selective undo. One operation may be undone concurrently, therefore, effect of multiple-undo needs to be preserved. Existing research has presented two alternative effects of multiundo [61]. One is that an operation is undone successfully if the number of undo commands is odd. Another is that an operation is undone by executing only one undo command. For example, in the literature [61], an undo command is interpreted as a concurrent inverse operation and a counter is used to record the total number of inverse operations related to the same undone

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

313

Fig. 3. v isible of all blocks in List after executing all operations.

Fig. 2. A collaborative scenario from the original state.

operation. In the literature [70], the line degree is used to record how many deletions are undone and redone. When the degree of a line is 1, the line appears in the document, otherwise, it does not appear in the document. In the literature [80], the number of concurrent undos of the same operation should be recorded. If the number of concurrent undos is an odd number, an operation can be successfully undone. In this paper, we define concurrent undos of the same operation as multiple undos, we use v isible to record the effect of multi-undo, which does not affect the total order of operation objects in the data structure. In order to explain the effect of multi-undo of the proposed method, a specific scenario of multi-undo is given as shown in Fig. 2. Three collaborative sites start from the same state ‘‘ ’’. Site1 generates the operation O1 that inserts ‘‘aaa’’ in the text. Then, O1 is broadcast to site2 and site3. Site2 and site3 generate operations O2 and O3 , which concurrently undo the same operation O1 . Then, site1 generates the operation O4 which deletes ‘‘aaa’’. After receiving O2 and O3 , site1 generates the operation O5 , which undos the operation O4 . Due to space limitation, site1 is selected as an example to explain the effect of multi-undo. At the beginning, the document is null. After O1 is executed in site1, the block ‘‘aaa’’ is inserted in List and Hash Table. According to Definition 4, the value of aaa.v isible is set to one. Then, O4 is executed in site1. According to Definition 4, the value of aaa.v isible is reduced by one and becomes zero. When O2 and O3 are received and executed in site1, according to Definition 5, the value of aaa.v isible is decreased by two and becomes negative two. At last, O5 is executed in site1, according to Definition 5, the value of aaa.v isible is increased by one and becomes negative one. After all operations have been executed in site1, v isible of all blocks in List is shown in Fig. 3. Meanwhile, when all operations have been executed in site2 and site3, Lists of site2 and site3 are the same as that of site1, such as v isible of all blocks and the total ordering relation of blocks. Finally, our undo-based control mechanism maintains the eventual consistency of shared document and preserves the operation intentions. How the system works and achieves eventual consistency are shown in Section 4. And the detailed formal proofs of correctness and intentions preservation are provided in Appendixes. Of course, this paper can also support the semantics of undo similarly as that in the literatures [70] and [80] with flexible modifications. The following gives how to modify our undo to support the semantics of undo in literatures [70] and [80].

the document. When a line is created, the degree of a line is 1. Each time a line is deleted, its degree is decreased by one. When an insertion is redone (a deletion is undone), the line degree is increased by one. A line is visible and appears in the document when its degree is 1. Otherwise, the line is invisible and does not appear in the document. In this paper, v isible is used to record how many times a block is deleted and reinserted (a deletion is undone). When a new block is inserted, v isible is set to 1. When a block is deleted, v isible is reduced by one. When a deletion is undone, v isible is increased by one. When an insertion is undone, v isible is decreased by one. Moreover, v isible is used to determine whether a block is visible or invisible in User −View . Based on the above discussions, our v isible is an alternative solution for line degree in the literature [70]. Therefore, v isible can be used instead of line degree to support the semantics of undo in the literature [70]. • In the literature [80], the number of concurrent undos of the same operation should be recorded to judge whether an operation can be successfully undone. When the number of concurrent undos is an odd number, an operation can be successfully undone. In order to support the semantics of undo in the literature [80], a new counter is needed to record the number of concurrent undos of the same operation. And the effect of selective undo should be redefined. Meanwhile, the algorithms of integrating (local/remote) undo operations should be modified. Detail modifications are shown as follows. (1) Firstly, in Definition 1, a new element counter is added in an operation O. O is a seven-tuple where counter records how many times an operation is undone. (2) Secondly, in Definition 5, the effect of selective undo needs to be redefined. A deletion can be successfully undone if its counter is an odd number, then block.v isible = block.v isible + 1. An insertion can be successfully undone if its counter is an odd number, then block.v isible = block. v isible − 1. (3) Thirdly, the algorithms of integrating (local/remote) undo operations need to be modified. We need to add a precondition to judge whether an undo operation is enable to be undone. Therefore, before a (local/remote) undo operation is integrated, counter of the target operation to be undone should be firstly got. If its counter is an odd number, the operation is enable to be undone, otherwise, it is not allowed to be undone. Definition 6 (The Intention of an Operation). Assume that the block is the object of an operation, the intention of an operation is defined as the position relation of the block in the single linked list(List).

• In the literature [70], a line degree is associated to each line. It records how many deletions are undone or redone. Meanwhile it indicates whether a line must appear or not in

In List, all blocks are linked from head to tail, the following gives the definition of position relations of blocks.

314

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

Definition 7 (block ≺). Given any two blocks in List, blocki and blockj . The positions of blocki and blockj are denoted as p(blocki ) and p(blockj ). If blocki ≺ blockj , iff : p(blocki ) < p(blockj ) (blocki is before blockj in List). Generally, when concurrent operations are generated, we can get block ≺ directly due to their different operation positions. But in some cases, two blocks have the same operation position in List without knowledge of each other. Here, we adopt the same method proposed in RGA, the unique identifier S4v ector of each block can order them. In order to preserve operation intentions and ensure eventual consistency, we need to make sure that all blocks in List have the same total ordering relation in every site when all operations have been executed in all sites. The following gives the definition of total ordering relations of all blocks. Definition 8 (Total Ordering Relations of Blocks(block ⇒)). Given any three blocks, blocki , blockj and blockk , then block⇒ exists, iff: (1) blocki ≺blockj , or blockj ≺blocki , or (2) blocki ≺blockj , blockj ≺blockk , then blocki ≺blockk . Definition 9 (Consistency of Operation Intentions). Assume that there are m collaborative sites, let L ={List1 , List2 , . . . , Listm }, List1 , List2 , . . . , Listm are single linked lists, which are respectively maintained by m sites. Suppose that there are n blocks in each Listi

List. For any two Lists, Listi and Listj in L, block ⇒ represents the Listj

total ordering relations of Listi , block ⇒ represents the total ordering relations of Listj . When all operations have been executed in m Listi

sites, consistency of operation intentions is achieved iff: block ⇒ Listj

= block ⇒ .

Algorithm 1 IntegrateLocalIns(O) 1: 2: 3: 4: 5: 6: 7: 8: 9:

Algorithm 3 gives how to integrate a local undo of the executed operation. In line 1, the target block is found using the hash function and tar_key. When the operation to be undone is an insertion, Tar_block.v isible is directly reduced by 1 (lines 2–3). When the operation to be undone is a deletion, Tar_block.v isible is directly increased by 1 (lines 4–5). When the operation to be undone is an undone operation, the target insertion (deletion) is found recursively (line 7). At last, a remote undo operation is generated and broadcast to remote sites (line 9). Algorithm 3 IntegrateLocalUndo(O) 1: 2: 3:

3.3. The algorithm integrated local operations The proposed algorithm handles the following three types of local operations. (1) IntegrateLocalIns(O), the parameter O is the local insert operation. (2) IntegrateLocalDel(O), the parameter O is the local delete operation. (3) IntegrateLocalUndo(O), the parameter O is an executed operation stored in the operation history, it may be a local operation or a received remote operation. Algorithm 1 presents how to integrate a local insertion. Different from our previous works [32], when a new block is inserted after the target block, there is no need to consider the spitting cases of the target block because the operation granularity is the atomic object. Therefore, given the unique identifier of the target block, the target block of List is found by using hash function. In line 2, the target block (Tar_block) is found by using the hash function and the unique identifier tar_key of the target block. When the target block is head of List, New _block is inserted after head (lines 3–4). When the target block is not head, New _block is inserted after Tar_block (lines 5–6). At last, New _block is placed into Hash Table and a remote insertion is generated and broadcast to remote sites (lines 8–9). Algorithm 2 presents how to integrate a local deletion. Different from our previous works [32], when a block is deleted, there is no necessity to deal with the splitting cases of the block because the operation granularity is the atomic object. For example, the target block is found by using the hash function and tar_key of the target block (line 1). Then, Tar_block.v isible is directly reduced by 1 (line 2). At last, a remote deletion is generated and broadcast to remote sites (line 3). Algorithm 2 IntegrateLocalDel(O) 1: 2: 3:

Tar_block=hash(O.tar_key) Tar_block.v isible=Tar_block.v isible − 1 a remote deletion is generated and broadcast to remote sites

New _block=new block(O.cur_key,O.con) Tar_block=hash(O.tar_key) if Tar_block is head then insert New _block after head else insert New _block after Tar_block end if place New _block into Hash Table a remote insertion is generated and broadcast to remote sites

4: 5: 6: 7: 8: 9:

Tar_block=hash(O.tar_key) if O.type==insert then Tar_block.v isible=Tar_block.v isible − 1 else if O.type==delete then Tar_block.v isible=Tar_block.v isible + 1 else IntegrateLocalUndo(O.tar_o) end if a remote undo of the operations is generated and broadcast to remote sites

3.4. The algorithm integrated remote operations

The proposed algorithm handles the following three types of remote operations. (1) IntegrateRemoteIns(O), the parameter O is a received remote insertion. (2) IntegrateRemoteDel(O), the parameter O is a received remote deletion. (3) IntegrateRemoteUndo(O), the parameter O is a received remote undo operation. Algorithm 4 presents how to integrate a remote insertion. In lines 1–2, a new block (New _block) is created and the target block (Tar_block) is found. Some concurrent insertions may have already inserted their new blocks next to Tar_block. Therefore, it is necessary to scan the blocks next to Tar_block until a block that its cur_key≺New _block.cur_key is first encountered (lines 9–16). Once such a block is found, New _block is inserted before it (lines 17). Algorithm 5 presents how to integrate a remote deletion. The target block (Tar_block) is found using tar_key (line 1). In line 2, Tar_block.v isible is directly reduced by 1.

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

315

Fig. 4. A scenario of integrating do/undo operations.

Algorithm 4 IntegrateRemoteIns(O) 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18:

New _block = new block(O.cur_key,O.con) Tar_block=hash(O.tar_key) if Tar_block is head then pre_block=head else pre_block=Tar_block end if next_block=pre_block.next while next_block ! = null do if O.cur_key>next_block.cur_key then break else pre_block = next_block next_block = next_block.next end if end while insert New _block between pre_block and next_block place New _block into Hash Table

Algorithm 5 IntegrateRemoteDel(O) 1: 2:

Tar_block=hash(O.tar_key) Tar_block.v isible=Tar_block.v isible − 1

Algorithm 6 presents how to integrate a remote undo of the executed operation. The processing procedure is similar to that of Algorithm 3. Algorithm 6 IntegrateRemoteUndo(O) 1: 2: 3: 4: 5: 6: 7: 8:

Tar_block=hash(O.tar_key) if O.type==insert then Tar_block.v isible=Tar_block.v isible − 1 else if O.type==delete then Tar_block.v isible=Tar_block.v isible + 1 else IntegrateRemoteUndo(O.tar_o) end if

3.5. The synchronization algorithm between User −View and List Algorithm 7 presents how to synchronize between User −View and List. In line 1, we get the first block of List. Then, we scan blocks of List one by one. We place the block into User −View if the block is not a tombstone (lines 2–6). 4. An integrated do/undo example Fig. 4 shows a specific scenario to illustrate how to integrate do/undo operations. Assume that the session number is 1 and

Algorithm 7 Synchronizing(List) 1: 2: 3: 4: 5: 6: 7:

block = List .head.next while (block != null) do if block.IsVisible() then User −View .Add(block) end if block=block.next end while

site number has such relation: site1
316

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330 Table 3 The average time complexity for our approach(ORGAU), RGA and Logoot. Algorithms Local Ins Logoot RGA ORGAU

Remote Del

UndoIns

UndoDel

Ins

Del

O(k) O(1) O(klog(n)) O(klog(n)) O(klog(n)) Best: O(1) O(N) O(N) Worst: O(m) Best: O(1) O(1) O(1) O(1) O(1) Worst: O(m)

UndoIns

UndoDel

O(klog(n)) O(klog(n)) O(klog(n)) O(1) O(1)

O(1)

O(1)

state vector, when the number of collaborative sites increases, the size of S4vector increases. 6. Experimental evaluation

Fig. 5. The final Hash Table, List and User −View after executing all do/undo operations.

of integrating a remote insertion is O(1). In this case, there are no concurrent inserted blocks after the target block, therefore, the new block is inserted after the target block directly once the target block is found. Suppose that the average number of concurrent inserted blocks is m, the worst-case time complexity of integrating a remote insertion is O(m) in case that there are m concurrent inserted blocks after the target block. Therefore, it is necessary to scan the concurrent inserted blocks to find the correct operation position. As specified in Algorithms 5 and 6, the time complexity of integrating a remote deletion and a remote undo operation are both O(1) as target blocks are found directly via Hash Table using the unique S4v ectors. Table 3 shows the average time complexity for our approach (ORGAU), RGA and Logoot. The notation k is the size of the line identifier, n is the number of lines in the shared document and N is the number of blocks in the shared document. N is equal to n plus the number of tombstones. According to the discussion in the literature [69,70], the time complexity of integrating a local insertion and a local deletion are respectively O(k) and O(1). The time complexity of integrating a remote insertion and a remote deletion are both O(klog(n)) as the binary search algorithm is used to find a position in Logoot. Therefore, it makes at most O(log(n)) comparisons, and each comparison costs O(k). Therefore, its time complexity is O(klog(n)). The time complexity of integrating a local undo operation and a remote undo operation are both O(klog(n)) [70]. In this case, Logoot needs at most O(log(n)) comparisons and each comparison costs O(k). According to the discussion in RGA [48], the time complexity of a local insertion and a local deletion are both O(N) as target blocks need to be searched via the single linked list from head. As shown in Table 3, ORGAU outperforms Logoot and RGA greatly. In the meantime, space complexity is also a significant measure criteria. For most CRDT algorithms, the size of identifiers generally affect the space complexity. According to the discussion in the literature [70], the space complexity of Logoot-undo is O(kn). The value of k increases when each new line is inserted in the document. In the worst case, the space complexity is O(N 2 ) [70]. Compared with Logoot-undo, ORGAU adopts S4vector as the unique identifier, which is the same as that of RGA [48]. Therefore, ORGAU has the same space complexity with that of RGA in theory. Assume that the size of S4vector is l, the space complexity is O(lN). The size of S4vector is fixed and does not increase when each new line is inserted. But S4vector uses the

In this section, we more focus on the time complexity of our approach. We perform some experiments on our approach to verify if it actually works as the theoretical analysis. We compare the computing performance of our approach with that of RGA and Logoot-do/undo [48,69,70]. Because RGA has the best average computing performance in representative CRDT algorithms and Logoot-undo is the typical CRDT algorithm supporting selective undo [48,69,70]. All three algorithms are implemented in C# language and compiled by Visual Studio 2010 on windows 7 system, Intel Core i7, 3.60 GHz CPU. Generally, performance evaluations build on the operation logs generated by collaboration. Until now, there are no available public data sets on the collaborative editing systems because the operation logs of some commercial real-time collaboration systems such as Google Docs are service, the operation logs are not complete and freely available [1]. For example, Google server has published serialized operation logs by using the operational transforming algorithm of Jupiter system [39]. However, some real-time information cannot be obtained from public serialized logs, such as state vectors. Therefore, it is difficult to use public serialized logs to replay the real-time collaboration for experiments. Therefore, we design the real-time collaboration in order to obtain the operation logs. Then, replay all operations on the three algorithms. 6.1. Design of collaborative procedure In this section, we design the real-time collaborative procedure, which refers to the method in the literature [1]. In order to get real-time information, such as state vectors, we have modified and re-implemented all codes from public sources of TeamEdit4 in C# language and compiled by Visual Studio 2010. We design two collaborations to evaluate the computing performance of RGA, ORGAU and Logoot. The first collaboration is used to evaluate performance behaviors of RGA and ORGAU. To test the scalability of RGA and ORGAU, students are divided into five groups. The first group is composed of 2 students. The second group is composed of 4 students. The third group is composed of 6 students. The fourth group is composed of 8 students. The fifth group is composed of 10 students. Students of each group collaboratively write a document simultaneously in the same session. Every student of each group works on one computer. According to [70], the operation granularity is a line of characters, therefore, the operation granularity of both ORGAU and RGA are lines of characters. Therefore, each student is allowed to insert (delete) a line from Wikipedia pages.5 Because RGA cannot support selective undo, each student is not permitted undo any operation. In order to ensure rationality, validity of the integrated procedure, after every computer of each group 4 http://teamedit.sourceforge.net/. 5 http://en.wikipedia.org/wiki/Wikipedia:Most_frequently_edited_pages.

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

317

Table 4 The average execution time of a local operation for RGA and ORGAU(µs). The number of local operations varies from 1000 to 10 000 with step 1000 Algorithms Del/Total = 20%

RGA ORGAU

Del/Total = 40%

Ins

Del

Ins

Del

50.08 0.86

32.53 0.59

36.97 0.91

15.59 0.45

executing all local operations and remote operations issued by their group, the same procedure continues with 25 times. Then the average execution time is recorded by students. The second collaboration is used to test computing performance of ORGAU and Logoot-do/undo. To evaluate the scalability of ORGAU and Logoot-do/undo, students are divided into five groups. The number of students of each group and how they work are the same as that of the first collaboration. Different from the first collaboration, Logoot supports selective undo, therefore, each student is permitted to undo any operation in the operation history. After all (do/undo) operations have been integrated in each computer of each group, the same procedure continues with 25 times and the average execution time is recorded. Based on the ReplicationBenchmark revealed on GitHub platform,6 we have reimplemented RGA, Logoot-do/undo and developed ORGAU algorithm in C# language. All three algorithms are compiled by Visual Studio 2010. In experiments, we use C#.Stopwatch class to test the execution time of each line-based operation.

Fig. 6. The computing time of a local operation when the delete ratio is set to 20%.

6.2. Comparison between RGA and ORGAU We compare the average computing time of our approach with that of RGA [48] when local (remote) insertions (deletions) are integrated. Meanwhile, we test the scalability of ORGAU and RGA when the number of collaborative sites increases. Table 4 shows the average computing time of a local insertion (deletion) for our approach (ORGAU) and RGA when the deletion ratio are respectively set to 20% and 40%. As shown in Table 4, the average execution time of ORGAU is much lower than that of RGA. The main reason is that ORGAU integrates a local insertion (deletion) with the time complexity of O(1). However, RGA integrates a local insertion (deletion) with the time complexity of O(N), where N is the number of blocks. With the number of operations increases, more blocks are inserted in the document. Therefore, RGA costs more time to integrate a local insertion (deletion). The results we obtained are conforming to the average theoretical complexities. Apparently, ORGAU outperforms RGA when integrating a local insertion (deletion). Figs. 6 and 7 show the average computing time of a local insertion (deletion) for RGA and ORGAU when the deletion ratios are set to 20% and 40%. By comparison, the average computing time of a local insertion (deletion) for RGA increases with the number of operations increases. However, the average execution time of a local insertion (deletion) for ORGAU decreases. The main reason is that the execution time of a local insertion (deletion) for RGA depends on the number of blocks in the document. When more and more operations are integrated, the number of blocks in the document becomes large, therefore, RGA needs to spend more time to find the operation position and integrates an operation. For ORGAU, the execution time of a local insertion (deletion) does not rely on the number of blocks. Given the unique identifier of the target block, ORGAU finds the target block with the time complexity of O(1). In other words, even the number of 6 http://github.com/PascalUrso/ReplicationBenchmark.

Fig. 7. The computing time of a local operation when the delete ratio is set to 40%.

operations increases, the execution time of a local operation for ORGAU will not increase. Because the increase rate of the number of operations exceeds the increase rate of total execution time of a batch of operations, the average execution time of a local insertion (deletion) for ORGAU decreases when the number of operations increases with step 1000. As shown in Figs. 6 and 7, ORGAU outperforms RGA when integrating a local insertion (deletion). Table 5 shows the average computing time of a remote insertion (deletion) for ORGAU and RGA when the deletion ratios are respectively set to 20% and 40%. As shown in Table 5, the average execution time of a remote insertion (deletion) for ORGAU is almost the same as that of RGA because they have the same time complexity when integrating a remote insertion (deletion). The experiment results we obtained are consistent with the average theoretical complexities. Observed from experiment results in Table 5, the execution time of a remote insertion for ORGAU and RGA is higher than that of a remote deletion. The main reason is that the time complexity of a remote insertion for RGA and ORGAU is higher than that of a remote deletion. For example, according to the discussions in Section 5, when integrating a remote insertion, both RGA and ORGAU have the time complexity

318

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

Table 5 The average execution time of a remote operation for RGA and ORGAU(µs). The number of remote operations varies from 1000 to 10 000 with step 1000 Algorithms Del/Total = 20%

RGA ORGAU

Del/Total = 40%

Ins

Del

Ins

Del

1.652 1.648

0.62 0.637

1.628 1.638

0.605 0.607

Fig. 9. The computing time of a remote operation when the delete ratio is set to 40%.

Fig. 8. The computing time of a remote operation when the delete ratio is set to 20%.

of O(m) in case that there are m concurrent insertions have the same inserted position. However, when integrating a remote deletion, both RGA and ORGAU have the time complexity of O(1) as the target deleted block is found directly via Hash Table using the unique identifier. Figs. 8 and 9 show the average computing time of a remote insertion (deletion) for RGA and ORGAU when the deletion ratios are set to 20% and 40%. By comparison, when the number of operations increases, the average computing time of a remote insertion for RGA is almost the same as that of ORGAU. For example, when the deletion ratio is 20% and the number of operation is 10 000, RGA takes about 1.29 µs to integrate a remote insertion and ORGAU takes about 1.3 µs to integrate a remote insertion. In addition, when the number of operations increases, the average computing time of a remote deletion for RGA is almost the same as that of ORGAU. For example, when the deletion ratio is 40% and the number of operation is 10 000, RGA takes around 0.47 µs to integrate a remote deletion and ORGAU takes about 0.49 µs to integrate a remote deletion. As shown in Table 5, Figs. 8 and 9, when integrating a remote operation, the computing performance of RGA and ORGAU are similar. The experiment results verify the correctness of theoretical time complexities. Figs. 10 and 11 show the average computing time of a local insertion (deletion) for RGA and ORGAU when the deletion ratio is set to 20% and the number of collaborative sites varies from 2 to 10 with step 2. As shown in Figs. 10 and 11, when the number of operations is fixed, with the number of sites increases, the execution times of a local insertion (deletion) for RGA and ORGAU increase. For example, in Fig. 10, when the number of operations is 10000 and the number of sites varies from 2 to 10, the execution times of a local insertion for RGA are respectively 90.63 µs, 101.19 µs, 108.76 µs, 114.71 µs and 120 µs. For ORGAU, the execution times of a local insertion are respectively 0.59 µs, 0.65 µs, 0.87 µs, 1.2 µs and 1.41 µs. The reason why

Fig. 10. The computing time of a local insertion when the delete ratio is set to 20% and the number of sites increases from 2 to 10 with step 2.

the execution times for RGA and ORGAU increase with growing numbers of sites is that RGA and ORGAU use state vectors to define the identifier. When the number of sites grows, the size of state vectors increases, therefore, the size of identifier grows also. When a conflict happens between concurrent operations, it costs more time to compare identifiers of conflicting operations. As shown in Figs. 10 and 11, although the execution times of a local insertion (deletion) for RGA and ORGAU increase, the increase rates are small with growing numbers of sites. Overall, the scalability of RGA and ORGAU are good when the number of sites increases. 6.3. Comparison between logoot and ORGAU We compare the average computing time of ORGAU with that of Logoot-do/undo [69,70] when local and remote (do/undo) operations are integrated. In addition, we evaluate the scalability of ORGAU and Logoot-do/undo. Table 6 shows the average computing time of a local (remote) insertion (deletion) for ORGAU and Logoot when the deletion ratio is set to 20%. As shown in Table 6, ORGAU overwhelms Logoot when integrating local (remote) insertions (deletions). In

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

Fig. 11. The computing time of a local deletion when the delete ratio is set to 20% and the number of sites increases from 2 to 10 with step 2. Table 6 The average time complexity of an insertion (a deletion) for Logoot and ORGAU(µs). The number of operations varies from 1000 to 10 000 with step 1000 Del/Total = 20% Algorithms Local

Logoot ORGAU

Remote

Ins

Del

Ins

Del

3.96 0.86

1.27 0.59

6.37 1.69

7.39 0.64

particular, when integrating a remote insertion (deletion), ORGAU outperforms Logoot by orders of magnitudes. The main reason is that ORGAU and Logoot have different time complexities when integrating a local (remote) operation. For example, the time complexity of a remote insertion (deletion) for Logoot is O(klog(n)), where n is the number of lines and k is the size of the line identifier. With growing numbers of operations, n and k become more and more large, therefore, Logoot costs more time to integrate a remote insertion (deletion). For ORGAU, the time complexity of a remote insertion (deletion) is O(1). The worstcase time complexity of a remote insertion is O(m), where m is the number of concurrent conflicting operations that they have the same inserted position. If there are no concurrent conflicting operations, even the number of operations increases, the average execution time of a remote insertion (deletion) for ORGAU does not increase. Of course, if all operations are concurrent conflicting operations, the execution time of a remote insertion for ORGAU will increase with growing numbers of operations. But the worst case will not often happen. Therefore, ORGAU has better computing performance than that of Logoot when integrating local (remote) operations. Fig. 12 shows the average computing time of a local (remote) insertion for ORGAU and Logoot. As shown in Fig. 12, ORGAU takes lower execution time than Logoot when local (remote) insertions are integrated. For example, when the number of operations increases, Logoot takes around 3.96 µs and 6.37 µs to integrate a local insertion and a remote insertion. The main reason why ORGAU outperforms Logoot is that ORGAU has lower time complexities than Logoot when integrating local (remote) insertions. We also observe that the execution time of a remote insertion for Logoot is higher than that of a local insertion. The reason is that the execution time of a local insertion for Logoot depends on the size of the line identifier k, and the execution time of a remote insertion relies on k and the number of lines n. Therefore, when the number of operations increases, more lines are inserted in the document, Logoot takes more time to

319

Fig. 12. The computing time of an insertion when the delete ratio is set to 20%.

integrate a remote insertion than a local insertion. For ORGAU, the execution time of a remote insertion is higher than that of a local insertion. The reason is that the time complexity of a local insertion is O(1). However, the best-case time complexity of a remote insertion is O(1) and the worst-case time complexity is O(m). With growing numbers of operations, there may exist more and more concurrent conflicting operations, therefore, the execution time of a remote insertion is higher than that of a local insertion. Fig. 13 shows the average computing time of a local (remote) deletion for ORGAU and Logoot. With the number of operations increases, Logoot takes about 6 µs to integrate a remote deletion, and it takes around 1.27 µs to integrate a local deletion. Compared with Logoot, ORGAU takes around 0.64 µs to integrate a remote deletion, and it takes 0.59 µs to integrate a local deletion. As shown in Fig. 13, ORGAU has better computing performance than Logoot when integrating local (remote) deletions. The main reason is that ORGAU has lower time complexities than Logoot when integrating local (remote) deletions. For example, Logoot integrates a remote deletion with the time complexity of O(klog(n)), where k is the size of line identifier and n is the number of lines. When the number of operations increases, k and n become large, therefore, the execution time of a remote deletion for Logoot increases. For ORGAU, the time complexity of a remote deletion is O(1). The execution time of a remote deletion will not increase with growing numbers of operations. For Logoot, the time complexity of a local deletion is O(1). When a line is deleted, the complicated synchronization procedure of Document, Idtable and cemetery is needed to make the applied effects appear in Document. For ORGAU, the time complexity of a local deletion is O(1). The target deleted block is found directly via Hash Table using unique identifier and it is remained as a tombstone. Therefore, ORGAU has a simpler integrating procedure than that of Logoot. That is the reason why they have the similar time complexity but different execution time. Table 7 shows the average computing time of selective undo for ORGAU and Logoot when the deletion ratio is set to 20%. As shown in Table 7, ORGAU has lower execution time than Logoot when local and remote undo operations are integrated. The reason is that ORGAU has lower time complexities than Logoot when integrating local (remote) undo operations. For example, the time complexity of a local (remote) undo operation for ORGAU is O(1). However, the time complexity of a local (remote) undo operation for Logoot is O(klog(n)). With growing numbers of operations, k and n increase, therefore, the execution time of a local (remote) undo operation for Logoot increases.

320

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

Fig. 13. The computing time of a deletion when the delete ratio is set to 20%.

Fig. 14. The computing time of a local undo of any operation when the delete ratio is set to 20%.

Table 7 The average time complexity of selective undo for Logoot and ORGAU(µs). The number of operations varies from 1000 to 10 000 with step 1000 Del/Total = 20% Algorithms Local

Logoot ORGAU

Remote

UndoIns

UndoDel

UndoIns

UndoDel

8.55 0.51

7.79 0.57

8.69 0.53

7.90 0.60

Fig. 14 shows the average computing time of a local undo operation for ORGAU and Logoot. ORGAU overwhelms Logoot obviously when local undo operations are integrated. For example, when the number of operations is 8000, Logoot takes 10.1 µs to integrate a local undo of the insertion, and it takes 10.06 µs to integrate a local undo of the deletion. Compared with Logoot, ORGAU takes 0.41 µs to integrate a local undo of the insertion, and it takes 0.47 µs to integrate a local undo of the deletion. The reason why ORGAU has better computing performance is that ORGAU integrates a local undo operation with the time complexity of O(1). However, Logoot integrates a local undo operation with the time complexity of O(klog(n)). Therefore, when the number of operations increases, the execution time of a local undo operation for Logoot increases. Fig. 15 shows the average computing time of a remote undo operation for ORGAU and Logoot. As shown in Fig. 15, ORGAU outperforms Logoot greatly when remote undo operations are integrated. For example, Logoot takes around 8.69 µs to integrate a remote undo of the insertion, and it takes about 7.9 µs to integrate a remote undo of the deletion. Compared with Logoot, ORGAU takes around 0.53 µs to integrate a remote undo of the insertion, and it takes 0.6 µs to integrate a remote undo of the deletion. The reason why ORGAU outperforms Logoot is that ORGAU integrates a remote undo operation with the time complexity of O(1). However, Logoot integrates a remote undo operation with the time complexity of O(klog(n)). With growing numbers of operations, Logoot takes more time to integrate a remote undo operation. Figs. 16 and 17 show the average computing time of a local undo of an insertion (a deletion) for ORGAU and Logoot when the deletion ratio is set to 20% and the number of collaborative sites varies from 2 to 10 with step 2. As shown in Figs. 16 and 17, ORGAU outperforms Logoot with growing numbers of collaborative sites. For example, in Fig. 16, when the number of operations is 10000 and the number of sites varies from 2 to 10, the execution times of a local undo of an insertion for ORGAU

Fig. 15. The computing time of a remote undo of any operation when the delete ratio is set to 20%.

are 0.417 µs, 0.473 µs, 0.554 µs, 0.67 µs and 0.727 µs. For Logoot, when the number of operations is 10000 and the number of sites varies from 2 to 10, the execution times of a local undo of an insertion are 9.671 µs, 11.036 µs, 13.897 µs, 17.945 µs and 23.545 µs. By comparison, when the number of sites increases, the execution time of a local undo of an insertion for Logoot increases rapidly. The main reason is that the execution time of a local undo operation for Logoot depends on the size of identifier k and the number of lines n. When the number of sites grows and the number of operations increases, k and n grow fast. Therefore, the execution time of a local undo operation for Logoot grows rapidly. Observed from Figs. 16 and 17, ORGAU and Logoot have good scalability with the growth of the number of collaborative sites. Figs. 18 and 19 show the average computing time of a remote undo of an insertion (a deletion) for ORGAU and Logoot when the deletion ratio is set to 20% and the number of collaborative sites varies from 2 to 10 with step 2. As shown in Figs. 18 and 19 , ORGAU has better computing performance than Logoot with growing numbers of operations and collaborative sites. For example, in Fig. 18, when the number of operations is 7000 and the number of sites varies from 2 to 10, the execution times of a remote undo of an insertion for ORGAU are 0.44 µs, 0.503 µs, 0.556 µs, 0.628 µs and 0.638 µs. For Logoot, when the number

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

321

Fig. 16. The computing time of a local undo of an insertion when the delete ratio is set to 20% and the number of sites varies from 2 to 10 with step 2. Fig. 18. The computing time of a remote undo of an insertion when the delete ratio is set to 20% and the number of sites varies from 2 to 10 with step 2.

Fig. 17. The computing time of a local undo of a deletion when the delete ratio is set to 20% and the number of sites varies from 2 to 10 with step 2. Fig. 19. The computing time of a remote undo of a deletion when the delete ratio is set to 20% and the number of sites varies from 2 to 10 with step 2.

of operations is 7000 and the number of sites varies from 2 to 10, the execution times of a remote undo of an insertion are 9.919 µs, 15.12 µs, 23.059 µs, 28.11 µs and 39.938 µs. By comparison, when the number of sites increases, the execution time for Logoot increases by orders of magnitudes. The main reason is that the execution time of a remote undo of an insertion for Logoot depends on the size of the identifier k and the number of lines n. When the number of sites grows and the number of operations increases, k and n increase rapidly, therefore, the execution time of a remote undo of an insertion for Logoot increases rapidly. For ORGAU, the execution time of a remote undo of an insertion only relies on the number of sites because ORGAU use state vectors to define the identifier. Therefore, the execution time of a remote undo of an insertion increases slowly with growing numbers of

sites. Overall, ORGAU and Logoot have good scalability when the number of sites grows. 7. An overall scheme of cloud-based collaborative editing service With the popularity of big data and cloud computing, massivescale collaborative editing systems have to address a tremendous amount of computing power and data storage. However, personal computers or mobile devices always have constrained resources even though their computing systems increasingly advance. One feasible solution is to offload computation and data storage to the cloud. Therefore, cloud-based collaborative editing service

322

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

Fig. 20. The overall cloud-based collaborative framework.

is necessary in the coming days. Existing cloud-based collaborative editing approaches adopt the OT-based synchronization technique to maintain the consistency of the shared document in the cloud [35,37]. Here, based on ORGAU algorithm proposed in this paper, we present an overall solution of cloud-based collaborative editing service. In future work, we will refer to [35] and use the Apache Tomcat77 Web server to develop the proposed cloud-based collaborative editing service. 7.1. The cloud-based collaborative editing framework The overall system architecture of the cloud-based collaborative editing framework is illustrated in Fig. 20. The whole cloud-based collaborative editing framework consists of multiple front-end devices and the back-end cloud platform. Each front-end device maintains a local copy of the shared document DS and a full replicated system architecture SA. SA includes a hash table (Hash Table), a single linked list (List) and an operation history (Log). The uses of these data structures are the same as that of ORGAU presented in this paper. The back-end cloud platform stores copies of their frontend devices, including the system architecture replica SA′ and the shared document replica DS ′ . The core component of back-end cloud platform is Control Module. It involves three important components, Copy Management, Synchronization Mechanism and Communication Mode Builder.

• Copy Management module handles all aspects of copy creation, copy deletion, copy modification and copy storage and retrieval. In order to provide a full life cycle of copies, 7 http://tomcat.apache.org/.

a NoSQL key–value database is used to store all copies. The reason why we use a key–value database is that the database is very fast, highly scalable and very suitable for storing and querying big data [4]. • Synchronization Mechanism has two functions. Firstly, the consistency of back-end copies in the cloud should be ensured. We adopt ORGAU presented in this paper as the consistency maintenance approach, which is one kind of CRDT-based techniques and adaptable to large-scale collaborative environments. Secondly, the coherence between the front-end local copy and its back-end copy in the cloud should be preserved. Two factors lead to the inconsistency. One is that local operations issued by the front-end device are not transmitted immediately to its back-end copy in the cloud. Another is that remote operations are only integrated in its back-end copy in the cloud. When the front-end device sends the request for synchronization to the cloud platform, its back-end copy may be synchronizing with other backend copies in the cloud. In order to prevent the conflict, as long as the front-end device broadcasts its local operations to the cloud, its back-end copy in the cloud should integrate its local operations instantly, and remote operations from other copies need to be integrated later. • Communication Mode Builder is mainly used to create a private peer to peer network for back-end copies in the cloud. In addition, it offers a communication channel from front-end device to its back-end copy in the cloud. And it enables back-end copies in the cloud to access to external available services. 7.2. The whole control mechanism The overall control mechanism includes the front-end synchronization and the back-end synchronization.

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

For the front-end devices, they generate and integrate local operations, broadcast their local operations to the cloud, and synchronize with their back-end copies in the cloud. Because the front-end devices have limited resources, many heavy tasks are delegated to the back-end cloud platform. The back-end cloud platform executes more heavy computing tasks, such as integrating remote operations, synchronization with the front-end devices, the consistency maintenance of the shared document and building the private peer to peer network. The whole control procedure is as follows. When a new collaborative session begins, local copies of the front-end devices are synchronized with their back-end copies in the cloud. Then the front-end users freely generate and integrate local operations in their local copies. When the front-end devices want to synchronize with their back-end copies in the cloud. They send the request to the cloud platform and broadcast local operations to their back-end copies in the cloud. Then back-end copies in the cloud firstly integrate local operations from their front-end devices, and broadcast their front-end local operations to other copies in the cloud. Meanwhile, back-end copies receive and integrate remote operations from other copies by invoking the ORGAU algorithm proposed in this paper. Then the new document state DS ′ and the system architecture SA′ of their back-end copies are transmitted to their front-end devices. And the frontend local copies are updated with DS ′ and SA′ from the cloud. Then front-end devices begin to generate and integrate new local operations based on their updated local copies. 8. Conclusions and future directions Selective undo is an essential function in large-scale collaborative text editing systems. This paper presents an efficient CRDT algorithm to support selective undo of any operation. Theoretical analysis and experimental evaluation show that the computing performance of the proposed algorithm outperforms the typical CRDT algorithm supporting selective undo by orders of magnitudes. Meanwhile, the proposed algorithm preserves the operation intentions and achieves the eventual consistency. The correctness and consistency of operation intention have been formally proved. Finally, an overall scheme of cloud-based collaborative editing service is presented based on the proposed algorithm, which gives the direction for future work. In future research, we plan to extend this work at least four directions but not limited. Firstly, we will develop and implement the cloud-based collaborative editing service referred to [35]. Secondly, we will evaluate ORGAU overhead in space and explore how to reduce the space complexity. Thirdly, we will extend the idea to other areas of computer science, such as Image process [16,24,38,82], Video process [23,25,26,59] and CAD & Graphics & Social media [29,30,41,42,72,75]. Fourthly, we will explore how to accelerate massive-scale collaborative applications in multi-core/many-core parallel computing model [9,20,45,66, 71,78,79,86,87]. Acknowledgments

323

Appendix A. Formal proof of correctness We formalize two correctness criteria called PT (Precedence Transitivity) and OC (Operation Commutativity) for providing correctness verification. These two conditions are proposed in RGA which only supports of do. Here we extend the conditions to support of selective undo. ORGAU is correct if the following two formal conditions are always hold. (1) PT: Given two (do/undo) operations, O1 and O2 , O2 takes precedence over O1 , iff: O1 is happened-before O2 or O1 ∥O2 , O2 has higher priority than O1 in preserving their intentions. PT holds iff: given any three (do/undo) operations, O1 , O2 and O3 , if O1 takes precedence O2 , O2 takes precedence O3 , for O1 takes precedence O3 . (2) OC: Given two (do/undo) operations, O1 and O2 from the same state S, O1 and O2 are commutative, denoted as O1 ↔O2 , iff: O1 →O2

O2 →O1

S ⇒ S1 , S ⇒ S2 , S1 is equal to S2 (S1 =S2 ). Theorem 1. In our approach, all (do/undo) operations satisfy PT. For any three operations Oa , Ob and Oc , if Oa takes precedence Ob , Ob takes precedence Oc , then, Oa takes precedence Oc . Proof. In our approach, we adopt the same unique identifier S4v ector, which has been proposed in RGA. When every (do/undo) operation is issued, it is assigned the unique identifier S4v ector. S4v ector of every operation is unique and S4v ector orders are transitive, which have been proved in RGA. Therefore, PT holds in our approach. □ Theorem 2. For any two remote insertions Ia and Ib , if Ia ∥Ib , Ia ↔Ib . Proof. Assume that Ia and Ib start from the same initial state S = [block1 , block2 , . . . , blocki , . . . , blockn ]. The object of Ia is blocka , the object of Ib is blockb , S4v ectorblocka ≺S4v ectorblockb . We need to consider two cases: (1) The target block of Ia is the same as the target block of Ib , denoted as blocki . 1 At the initial state S, Ia is first executed, then, Ib is executed. ⃝ Ia is firstly executed at the initial state S, blocka needs to insert after blocki . Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockj ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockj . After executing Ia , S1′ = [block1 , block2 , . . . , blocki , . . . , blocka , blockj , . . . , blockn ]. Then, Ib is executed at the state S1′ . Because there exists S4v ectorblocka ≺ S4v ectorblockb , blockb needs to insert before blocka . In addition, concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblockb with S4v ectors of concurrent inserted blocks. Assume that a block(blockk ) whose S4v ector ≺ S4v ectorblockb is first encountered, therefore, blockb is inserted before blockk . In this case, two cases need to be considered further. (1) blockk is the same as blockj .

This research has been funded by the National Science Foundation of China (Grant Nos. 61472289 and 61502353) and the Natural Science Foundation of Hubei Province, China (Grant No. 2016YfFC010 6305).

Because there exists S4v ectorblocka ≺S4v ectorblockb , blockb is inserted before blocka . After executing Ib , S2′ = [block1 , block2 , . . . , blocki , . . . , blockb , blocka , blockj (blockk ), . . . , blockn ].

Declaration of competing interest

After executing Ib , S2′ = [block1 , block2 , . . . , blocki , . . . , blockb , blockk , . . . , blocka , blockj , . . . , blockn ]. 2 At the initial state S, Ib is first executed, then, Ia is executed. ⃝ Ib is first executed at the initial state S, blockb needs to insert after blocki . Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare

No author associated with this paper has disclosed any potential or pertinent conflicts which may be perceived to have impending conflict with this work. For full disclosure statements refer to https://doi.org/10.1016/j.jpdc.2019.05.005.

(2) blockk and blockj are different blocks.

324

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

S4v ectorblockb with S4v ectors of concurrent inserted blocks. Assume that a block(blockk ) whose S4v ector ≺ S4v ectorblockb is first encountered, therefore, blockb is inserted before blockk . After executing Ib , S1′′ = [block1 , block2 , . . . , blocki , . . . , blockb , blockk , . . . , blockn ]. Then, Ia is executed at the state S1′′ . Because there exists S4v ectorblocka ≺ S4v ectorblockb , blocka needs to insert after blockb . In addition, concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockj ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockj . In this case, two cases need to be considered further. (1) blockk is the same as blockj . Because there exists S4v ectorblocka ≺S4v ectorblockb , blockb is inserted before blocka . After executing Ia , S2′′ = [block1 , block2 , . . . , blocki , . . . , blockb , blocka , blockj (blockk ), . . . , blockn ]. (2) blockk and blockj are different blocks. After executing Ia , S2′′ = [block1 , block2 , . . . , blocki , . . . , blockb , blockk , . . . , blocka , blockj , . . . , blockn ]. ′ ′′ 1 (1) and ⃝ 2 (1), S2 is equal to S2 , Ia ↔Ib . From ⃝ ′ ′′ 1 (2) and ⃝ 2 (2), S2 is equal to S2 , Ia ↔Ib . From ⃝ (2) The target block of Ia is different from the target block of Ib . Assume that blocki is the target block of Ia , blockj is the target block of Ib . We need to consider two cases: (a) blocki is before blockj . 1 At the initial state S, Ia is first executed, then, Ib is executed. ⃝

Ia is first executed at the initial state S. Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S1′ = [block1 , block2 , . . . , blocki , . . . , blocka , blockp , . . . , blockj , . . . , blockn ]. Then, Ib is executed at the state S1′ . Because concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblockb with S4v ectors of concurrent inserted blocks. Assume that a block(blockq ) whose S4v ector ≺ S4v ectorblockb is first encountered, therefore, blockb is inserted before blockq . After executing Ib , S2′ = [block1 , block2 , . . . , blocki , . . . , blocka , blockp , . . . , blockj , . . . , blockb , blockq , . . . , blockn ]. 2 At the initial state S, Ib is first executed, then, Ia is executed. ⃝ Ib is first executed at the initial state S. Because concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblockb with S4v ectors of concurrent inserted blocks. Assume that a block(blockq ) whose S4v ector ≺ S4v ectorblockb is first encountered, therefore, blockb is inserted before blockq . After executing Ib , S1′′ = [block1 , block2 , . . . , blocki , . . . , blockj , . . . , blockb , blockq , . . . , blockn ]. Then, Ia is executed at the state S1′′ . Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S2′′ = [block1 , block2 , . . . , blocki , . . . , blocka , blockp , . . . , blockj , . . . , blockb , blockq , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ia ↔Ib . From ⃝ (b) blocki is after blockj . 1 At the initial state S, Ia is executed, then, Ib is executed. ⃝

Ia is first executed at the initial state S. Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S1′ = [block1 , block2 , . . . ,

blockj , . . . , blocki , . . . , blocka , blockp , . . . , blockn ]. Then, Ib is executed at the state S1′ . Assume that a block(blockq ) whose S4v ector ≺ S4v ectorblockb is first encountered, therefore, blockb is inserted before blockq . After executing Ib , S2′ = [block1 , block2 , . . . , blockj , . . . , blockb , blockq , . . . , blocki , . . . , blocka , blockp , . . . , blockn ]. 2 At the initial state S, Ib is first executed, then, Ia is executed. ⃝ Ib is first executed at the initial state S. Because concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblockb with S4v ectors of concurrent inserted blocks. Assume that a block(blockq ) whose S4v ector ≺ S4v ectorblockb is first encountered, therefore, blockb is inserted before blockq . After executing Ib , S1′′ = [block1 , block2 , . . . , blockj , . . . , blockb , blockq , . . . , blocki , . . . , blockn ]. Then, Ia is executed at the state S1′′ . Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S2′′ = [block1 , block2 , . . . , blockj , . . . , blockb , blockq , . . . , blocki , . . . , blocka , blockp , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ia ↔Ib . □ From ⃝ Theorem 3. For a remote insertion Ia and a remote deletion Da , if Ia ∥Da , Ia ↔Da . Proof. Assume that Ia and Da start from the same initial state S = [block1 , block2 , . . . , blockn ]. The value of each block.v isible is initialed to 1. The target block of Ia is blocki . The object of Ia is blocka and the target block of Da is blockj . We need to consider two cases: (1) blocki is the same as blockj . 1 At the initial state S, Ia is first executed, then, Da is executed. ⃝ Ia is first executed at the initial state S. Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S1′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blocka , blockp , . . . , blockn ]. Then, Da is executed. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Then, blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. After executing Da , S2′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blocka , blockp , . . . , blockn ], blocki (blockj ) is preserved as a tombstone. 2 At the initial state S, Da is first executed, then, Ia is executed. ⃝ Da is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Then, blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. After executing Da , S1′′ = [block1 , block2 , . . ., —————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is preserved as a tombstone. Then, Ia is executed at the state S1′′ . Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S2′′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blocka , blockp , . . . , blockn ], blocki (blockj ). —————— ′ ′′ 1 ⃝ 2 , S2 = S2 , Ia ↔Da . From ⃝ (2) blocki and blockj are different blocks. (a) blocki is before blockj . 1 At the initial state S, Ia is first executed, then, Da is executed. ⃝ Ia is first executed at the initial state S. Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

before blockp . After executing Ia , S1′ = [block1 , block2 , . . . , blocki , . . . , blocka , blockp , . . . , blockj , . . . , blockn ]. Then, Da is executed. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Then, blockj .v isible = blockj .v isible − 1 = 0. After executing Da , S2′ = [block1 , block2 , . . . , blocki , . . . , blocka , blockp , . . . ,—— blockj , . . . , blockn ], blockj is preserved as a tombstone. 2 At the initial state S, Da is first executed, then, Ia is executed. ⃝ Da is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Then, blockj .v isible = blockj .v isible − 1 = 0. After executing Da , S1′′ = [block1 , block2 , . . . , blocki , . . ., blockj , . . . , blockn ], blockj is preserved as a tombstone. Then, —— Da is executed at the state S1′′ . Because concurrent insertions may have inserted their new blocks next to blocki , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S2′′ = [block1 , block2 , . . . , blocki , . . . , blocka , blockp , . . . ,—— blockj , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ia ↔Da . From ⃝ (b) blocki is after blockj . In this case, the proof procedure is the same as that of (a), therefore, Ia ↔Da . □ Theorem 4. For any two remote deletions Da and Db , if Da ∥Db , Da ↔Db . Proof. Assume that Da and Db start from the same initial state, S = [block1 , block2 , . . . , blockn ]. The value of each block.v isible is initialed to 1. The target block of Da is blocki , and the target block of Db is blockj . We need to consider two cases: (1) blocki is the same as blockj . 1 At the initial state S, Da is first executed, then, Db is ⃝ executed. Da is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . Then, blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. After blocki (blockj ), . . . , blockn ], executing Da , S1′ = [block1 , block2 , . . ., —————— blocki (blockj ) is preserved as a tombstone. Then, Db is executed. The target block(blockj ) is the same as blocki . Then, blocki (blockj ). v isible = blocki (blockj ).v isible − 1 = −1. After executing Db , S2′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blockn ], blocki (blockj ) is also preserved as a —————— tombstone and blocki (blockj ).v isible = −1. 2 At the initial state S, Db is first executed, then, Da is ⃝ executed. Db is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Then, blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. After executing Db , S1′′ = [block1 , block2 , . . ., —————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is preserved as a tombstone. Then, Da is executed. The target block(blocki ) is the same as blockj . Then, blocki (blockj ). v isible = blocki (blockj ).v isible − 1 = −1. After executing Da , S2′′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is also preserved as a tombstone and blocki (blockj ).v isible = −1. ′ ′′ 1 ⃝ 2 , S2 = S2 , Da ↔Db . From ⃝ (2) blocki and blockj are different blocks. (a) blocki is before blockj . 1 At the initial state S, Da is first executed, then, Db is ⃝ executed. Da is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . Then, blocki .v isible = blocki .v isible − 1 = 0. After executing Da ,

325

S1′ = [block1 , block2 , . . . ,—— blocki , . . . , blockn ], blocki is preserved as a tombstone. Then, Db is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . Then, blockj .v isible = blockj .v isible − 1 = 0. After executing Db , S2′ = [block1 , block2 , . . . ,—— blocki , . . . ,—— blockj , . . . , blockn ], blockj is preserved as a tombstone. 2 At the initial state S, Db is first executed, then, Da is ⃝ executed. Db is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Then, blockj .v isible = blockj .v isible − 1 = 0. After executing Db , S1′′ = [block1 , block2 , . . . ,—— blockj , . . . , blockn ], blockj is preserved as a tombstone. Then, Da is executed. The target block(blocki ) is found using the hash function and S4v ectorblocki . Then, blocki .v isible = blocki .v isible − 1 = 0. After executing Da , S2′′ = [block1 , block2 , . . . ,—— blocki , . . . ,—— blockj , . . . , blockn ], blocki is preserved as a tombstone. ′ ′′ 1 ⃝ 2 , S2 = S2 , Da ↔Db . From ⃝ (b) blocki is after blockj . In this case, the proof procedure is the same as that of (a), therefore, Da ↔Db . □ Theorem 5. For a remote undo operation Ua and a remote insertion Ia , if Ua ∥Ia , Ua ↔Ia . Proof. Assume that Ua and Ia start from the same initial state S = [block1 , block2 , . . . , blockn ]. The value of each block.v isible is initialed to 1. The target block of Ua is blocki , the target block of Ia is blockj and the object of Ia is blocka . We need to consider two cases: (1) blocki is the same as blockj . (a) The target operation of Ua is an insertion. 1 At the initial state S, Ua is first executed, then, Ia is executed. ⃝ Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. S1′ = [block1 , block2 , . . ., —————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is preserved as a tombstone. Then, Ia is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . Because concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block (blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . S2′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blocka , blockp , . . . , blockn ]. 2 At the initial state S, Ia is first executed, then, Ua is executed. ⃝ Ia is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Because

concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S1′′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blocka , blockp , . . . , blockn ]. Then, Ua is executed. The target block (blocki ) is found using the hash function and S4v ectorblocki . Then, blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0, blocki (blockj ) is preserved as a tombstone. After executing Ua , S2′′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blocka , blockp , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Ia . From ⃝ (b) The target operation of Ua is a deletion.

326

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

1 At the initial state S, Ua is first executed, then, Ia is executed. ⃝ Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki (blockj ).v isible = blocki (blockj ).v isible + 1 = 2. S1′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blockn ]. Then, Ia is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . Because concurrent insertions may have

inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . S2′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blocka , blockp , . . . , blockn ]. 2 At the initial state S, Ia is first executed, then, Ua is executed. ⃝ Ia is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Because concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S1′′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blocka , blockp , . . . , blockn ]. Then, Ua is executed. The target block (blocki ) is found using the hash function and S4v ectorblocki . Then, blocki (blockj ).v isible = blocki (blockj ). v isible + 1 = 2. After executing Ua , S2′′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blocka , blockp , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Ia . From ⃝ (2) blocki and blockj are different blocks. (a) blocki is before blockj . (1) The target operation of Ua is an insertion.

[block1 , block2 , . . . , blocki , . . . , blockn ]. Then, Ia is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . Because concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . S2′ = [block1 , block2 , . . . , blocki , . . . , blockj , . . . , blocka , blockp , . . . , blockn ]. 2 At the initial state S, Ia is first executed, then, Ua is executed. ⃝ Ia is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Because concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S1′′ = [block1 , block2 , . . . , blockj , . . . , blocka , blockp , . . . , blockn ]. Then, Ua is executed. The target block (blocki ) is found using the hash function and S4v ectorblocki . Then, blocki .v isible = blocki . v isible + 1 = 2. After executing Ua , S2′′ = [block1 , block2 , . . . , blocki , . . . , blockj , . . . , blocka , blockp , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Ia . From (2)⃝ (b) blocki is after blockj . In this case, the proof procedure is the same as that of (a), therefore, Ua ↔Ia . □ Theorem 6. For a remote undo operation Ua and a remote deletion Da , if Ua ∥Da , Ua ↔Da .

1 At the initial state S, Ua is first executed, then, Ia is executed. ⃝ Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. blocki , . . . , blockn ], blocki is preserved as S1′ = [block1 , block2 , . . ., —— a tombstone. Then, Ia is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . Because concurrent

Proof. Assume that Ua and Da start from the same initial state S = [block1 , block2 , . . . , blockn ]. The value of each block.v isible is initialed to 1. The target block of Ua is blocki , the target block of Da is blockj . We need to consider two cases: (1) blocki is the same as blockj .

insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted blocki , . . . , blockj , . . . , before blockp . S2′ = [block1 , block2 , . . . ,—— blocka , blockp , . . . , blockn ]. 2 At the initial state S, Ia is first executed, then, Ua is executed. ⃝ Ia is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . Because

1 At the initial state S, Ua is first executed, then, Da is ⃝ executed. Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. S1′ = [block1 , block2 , . . ., —————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is preserved as a tombstone. Then, Da is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . After executing Da , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = −1, blocki (blockj ) is preserved as a tombstone. S2′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blockn ]. 2 At the initial state S, Da is first executed, then, Ua is ⃝ executed. Da is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . After executing Da , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. S1′′ = [block1 , block2 , . . ., —————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is preserved as a tombstone. Then, Ua is executed. The target block (blocki )is found using the hash function and S4v ectorblocki . Then, blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = −1, blocki (blockj ) is preserved as a tombstone. After executing Ua , S2′′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Da . From ⃝

concurrent insertions may have inserted their new blocks next to blockj , it is necessary to compare S4v ectorblocka with S4v ectors of concurrent inserted blocks. Assume that a block(blockp ) whose S4v ector ≺ S4v ectorblocka is first encountered, therefore, blocka is inserted before blockp . After executing Ia , S1′′ = [block1 , block2 , . . . , blockj , . . . , blocka , blockp , . . . , blockn ]. Then, Ua is executed. The target block (blocki ) is found using the hash function and S4v ectorblocki . Then, blocki .v isible = blocki .v isible − 1 = 0, blocki is preserved as a tombstone. After executing Ua , S2′′ = [block1 , block2 , . . . ,—— blocki , . . . , blockj , . . . , blocka , blockp , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Ia . From (1)⃝ (2) The target operation of Ua is a deletion. 1 At the initial state S, Ua is first executed, then, Ia is executed. ⃝ Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki .v isible = blocki .v isible + 1 = 2. S1′ =

(a) The target operation of Ua is an insertion.

(b) The target operation of Ua is a deletion. 1 At the initial state S, Ua is first executed, then, Da is ⃝ executed.

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki (blockj ).v isible = blocki (blockj ).v isible + 1 = 2. S1′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blockn ]. Then, Da is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . After executing Da , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 1. S2′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blockn ]. 2 At the initial state S, Da is first executed, then, Ua is ⃝ executed. Da is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . After executing Da , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. S1′′ = [block1 , block2 , . . ., —————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is preserved as a tombstone. Then, Ua is executed. The target block (blocki )is found using the hash function and S4v ectorblocki . Then, blocki (blockj ).v isible = blocki (blockj ).v isible + 1 = 1. After executing Ua , S2′′ = [block1 , block2 , . . . , blocki (blockj ), . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Da . From ⃝ (2) blocki and blockj are different blocks. (a) blocki is before blockj . (1) The target operation of Ua is an insertion. 1 At the initial state S, Ua is first executed, then, Da is ⃝

executed. Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki .v isible = blocki .v isible − 1 = 0. S1′ = [block1 , block2 , . . . ,—— blocki , . . . , blockn ], blocki is preserved as a tombstone. Then, Da is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . After executing Da , blockj .v isible = blockj .v isible − 1 = 0, blockj is preserved as a blocki , . . . ,—— blockj , . . . , blockn ]. tombstone. S2′ = [block1 , block2 , . . . ,—— 2 At the initial state S, Da is first executed, then, Ua is ⃝ executed. Da is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . After executing Da , blockj .v isible = blockj .v isible − 1 = 0. S1′′ = [block1 , block2 , . . . ,—— blockj , . . . , blockn ], blockj is preserved as a tombstone. Then, Ua is executed. The target block (blocki ) is found using the hash function and S4v ectorblocki . Then, blocki .v isible = blocki .v isible − 1 = 0, blocki is preserved as a tombstone. After executing Ua , S2′′ = [block1 , block2 , . . . ,—— blocki , . . . ,—— blockj , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Da . From (1)⃝ (2) The target operation of Ua is a deletion. 1 At the initial state S, Ua is first executed, then, Da is ⃝ executed. Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki .v isible = blocki .v isible + 1 = 2. S1′ = [block1 , block2 , . . . , blocki , . . . , blockn ]. Then, Da is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . After executing Da , blockj .v isible = blockj .v isible − 1 = 0, blockj is preserved as a tombstone. S2′ = [block1 , block2 , . . . , blocki , . . ., —— blockj ,..., blockn ]. 2 At the initial state S, Da is first executed, then, Ua is ⃝ executed. Da is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . After executing Da , blockj .v isible = blockj .v isible − 1 = 0. S1′′ = [block1 , block2 , . . . ,—— blockj , . . . , blockn ], blockj is preserved as a tombstone. Then, Ua is executed. The target block (blocki ) is found using the hash function and S4v ectorblocki . Then, blocki .v isible = blocki .v isible + 1 = 2. After executing Ua , S2′′ = [block1 , block2 , . . . , blocki , . . . ,—— blockj , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Da . From (2)⃝

327

(b) blocki is after blockj . In this case, the proof procedure is the same as that of (a), therefore, Ua ↔Da . □ Theorem 7. For a remote undo operation Ua and a remote undo operation Ub , if Ua ∥Ub , Ua ↔Ub . Proof. Assume that Ua and Ub start from the same initial state S = [block1 , block2 , . . . , blockn ]. The value of each block.v isible is initialed to 1. The target block of Ua is blocki , the target block of Ub is blockj . We need to consider two cases: (1) blocki is the same as blockj . In this case, there are four cases to be considered. (a) The target operation of Ua is an insertion, the target operation of Ub is an insertion. (b) The target operation of Ua is an insertion, the target operation of Ub is a deletion. (c) The target operation of Ua is a deletion, the target operation of Ub is an insertion. (d) The target operation of Ua is a deletion, the target operation of Ub is a deletion. Note, however, that for space reasons here we only give the proof procedure of (a), the other proof procedures are similar to that of (a). 1 At the initial state S, Ua is first executed, then, Ub is ⃝ executed. Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. blocki (blockj ), . . . , blockn ], blocki (blockj ) S1′ = [block1 , block2 , . . ., —————— is preserved as a tombstone. Then, Ub is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . After executing Ub , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = −1, blocki (blockj ) is also preserved as a tombstone. S2′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blockn ]. 2 At the initial state S, Ub is first executed, then, Ua is ⃝ executed. Ub is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . After executing Ub , blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = 0. S1′′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blockn ], blocki (blockj ) is preserved as a tombstone. Then, Ua is executed. The target block (blocki )is found using the hash function and S4v ectorblocki . Then, blocki (blockj ).v isible = blocki (blockj ).v isible − 1 = −1, blocki (blockj ) is also preserved as a tombstone. After executing Ua , S2′′ = [block1 , block2 , . . . ,—————— blocki (blockj ), . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Da . From ⃝ (2) blocki and blockj are different blocks. In this case, there are the same four cases to be considered, which have been shown in (1)(a)–(1)(d). Due to space reasons, here we only give the proof procedure of (a), the other proof procedures are similar to that of (a). (1) blocki is before blockj . 1 At the initial state S, Ua is first executed, then, Ub is ⃝ executed. Ua is first executed at the initial state S. The target block(blocki ) is first found using the hash function and S4v ectorblocki . After executing Ua , blocki .v isible = blocki .v isible − 1 = 0. S1′ = [block1 , block2 , . . . ,—— blocki , . . . , blockn ], blocki is preserved as a tombstone. Then, Ub is executed. The target block(blockj ) is found using the hash function and S4v ectorblockj . After executing Ub ,

328

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330

blockj .v isible = blockj .v isible − 1 = 0, blockj is preserved as a tombstone. S2′ = [block1 , block2 , . . . ,—— blocki , . . . ,—— blockj , . . . , blockn ]. 2 At the initial state S, Ub is first executed, then, Ua is ⃝ executed. Ub is first executed at the initial state S. The target block(blockj ) is first found using the hash function and S4v ectorblockj . After executing Ub , blockj .v isible = blockj .v isible − 1 = 0. S1′′ = [block1 , block2 , . . . ,—— blockj , . . . , blockn ], blockj is preserved as a tombstone. Then, Ua is executed. The target block (blocki ) is found using the hash function and S4v ectorblocki . Then, blocki .v isible = blocki .v isible − 1 = 0, blocki is preserved as a tombstone. After executing Ua , S2′′ = [block1 , block2 , . . . ,—— blocki , . . . ,—— blockj , . . . , blockn ]. ′ ′′ 1 ⃝ 2 , S2 = S2 , Ua ↔Ub . From (1)⃝ (2) blocki is after blockj . In this case, the proof procedure is the same as that of (1), therefore, Ua ↔Ub . □ Appendix B. Formal proof of consistency of operation intentions Theorem 8. Assume that there are m collaborative sites, let L = {List1 , List2 , . . . , Listm }, List1 , List2 , . . . , Listm are single linked lists, which are respectively maintained by m sites. Suppose that there are n blocks in each List. For any two Lists, Listi and Listj , when all (do/undo) operations have been executed in m sites, our approach Listi

Listj

can ensure block ⇒ = block ⇒ . Proof. Based on the above proof of correctness in section 3.1, concurrent (do/undo) operations execute commutatively to guarantee eventual consistency. Therefore, for any two Listi and Listj maintained by two sites, when all operations have been executed in two sites, block1 , block2 , . . . , blockn of Listi and block1 , block2 , . . . , blockn of Listj have the same total order. Therefore, Listi

Listj

our approach can achieve block ⇒ = block ⇒ . As a result, our approach can achieve consistency of operation intentions. □ References [1] M. AhmedNacer, C. Ignat, G. Oster, H. Roh, P. Urso, Evaluating crdts for real-time document editing, in: Proceedings of the 11th ACM Symposium on Document Engineering, ACM, 2011, pp. 103–112. [2] L. Andr, S. Martin, G. Oster, C. Ignat, Supporting adaptable granularity of changes for massive-scale collaborative editing, in: Proceedings of the 9th IEEE International Conference Conference on Collaborative Computing: Networking, Applications and Worksharing, IEEE, 2013, pp. 50–59. [3] P. Arbenz, W. Gander, M. Oettli, The remote computation system, Parallel Comput. 23 (10) (1997) 1421–1428. [4] M.D. Assuno, R.N. Calheiros, S. Bianchi, M.A. Netto, R. Buyya, Big data computing and clouds: Trends and future directions, J. Parallel Distrib. Comput. 79 (2015) 3–15. [5] D.A. Bader, Petascale Computing: Algorithms and Applications, CRC press, 2007. [6] L. Briot, P. Urso, M. Shapiro, High responsiveness for group editing CRDTs, in: Proceedings of ACM International Conference on Supporting Group Work, ACM, 2016, pp. 51–60. [7] H. Casanova, A. Giersch, A. Legrand, M. Quinson, F. Suter, Versatile, scalable, and accurate simulation of distributed applications and platforms, J. Parallel Distrib. Comput. 74 (10) (2014) 2899–2917. [8] X. Chen, F. He, H. Yu, A matting method based on full feature coverage, Multimedia Tools Appl. 78 (9) (2019) 11173–11201. [9] G. Chen, G. Li, B. Wu, S. Pei, A gpu-based computing framework for cscw, in: Proceedings of 14th International Conference on Computer Supported Cooperative Work in Design (CSCWD), IEEE, 2010, pp. 100–103. [10] W. Chen, G. Xie, R. Li, Y. Bai, C. Fan, K. Li, Efficient task scheduling for budget constrained parallel applications on heterogeneous cloud computing systems, Future Gener. Comput. Syst. 74 (2017) 1–11. [11] M.I. Daoud, N. Kharma, A high performance algorithm for static task scheduling in heterogeneous distributed computing systems, Journal of Parallel and distributed computing 68 (4) 399–409.

[12] A.H. Davis, C. Sun, J. Lu, Generalizing operational transformation to the standard general markup language, in: Proceedings of the ACM 2002 Conference on Computer Supported Cooperative Work, ACM, 2002, pp. 58–67. [13] A. Deftu, J. Griebsch, A scalable conflict-free replicated set data type, in: Proceedings of the 33rd International Conference on Distributed Computing Systems (ICDCS), IEEE, 2013, pp. 186–195. [14] P. Dewan, Inferred awareness to support mixed-activity collaboration, in: Proceedings of the 2nd International Conference on Collaboration and Internet Computing, IEEE, 2016, pp. 339–348. [15] Z. Du, H. Sun, Y. He, Y. He, D.A. Bader, H. Zhang, Energy-efficient scheduling for best-effort interactive services to achieve high response quality, in: Proceedings of the 27th International Symposium on Parallel & Distributed Processing, IEEE, 2013, pp. 637–648. [16] L. Gao, F. Yu, Q. Chen, N. Xiong, Consistency maintenance of do and undo/redo operations in real-time collaborative bitmap editing systems, Cluster Comput. 19 (1) (2016) 255–267. [17] N. Gu, J. Yang, Q. Zhang, Consistency maintenance based on the mark & retrace technique in groupware systems, in: Proceedings of the 2005 International ACM SIGGROUP Conference on Supporting Group Work, ACM, 2005, pp. 264–273. [18] H. Gu, Q. Zhang, B. Shao, Making autocad collaborative: implementation and application of coautocad, in: Proceedings of the 2nd International Conference on Pervasive Computing and Applications, IEEE, 2007, pp. 168–173. [19] N. Gu, Q. Zhang, J. Yang, W. Ye, Dcv: a causality detection approach for large-scale dynamic collaboration environments, in: Proceedings of the 2007 International ACM Conference on Supporting Group Work, ACM, 2007, pp. 157–166. [20] N. Hou, X. Yan, F. He, A survey on partitioning models, solution algorithms and algorithm parallelization for hardware/software co-design, Des. Autom. Embedded Syst. (2019) 1–21. [21] T. Kosar, M. Livny, A framework for reliable and efficient data placement in distributed computing systems, Journal of Parallel and Distributed Computing 65 (10) 1146–1157. [22] L. Lamport, Time, clocks, and the ordering of events in a distributed system, Commun. ACM 21 (7) (1978) 558–565. [23] K. Li, F. He, Yu. H., Chen X., A parallel and robust object tracking approach synthesizing adaptive bayesian learning and improved incremental subspace learning, Front. Comput. Sci. (2017) http://dx.doi.org/10.1007/ s11704-018-6442-4. [24] H. Li, F. He, X. Yan, Ibea-svm: An indicator-based evolutionary algorithm based on pre-selection with classification guided by svm, Appl. Math.-A J. Chin. Univ. 34 (1) (2019) 1–26. [25] K. Li, F. He, H. Yu, Robust visual tracking based on convolutional features with illumination and occlusion handling, J. Comput. Sci. Tech. 33 (1) (2018) 223–236. [26] K. Li, F. He, H. Yu, X. Chen, A correlative classifiers approach based on particle filter and sample set for tracking occluded target, Appl. Math.-A J. Chin. Univ. 32 (3) (2017) 294–312. [27] D. Li, R. Li, An admissibility-based operational transformation framework for collaborative editing systems, Comput. Support. Coop. Work(CSCW) 19 (1) (2010) 1–43. [28] R. Li, D. Li, C. Sun, A Time Interval Based Consistency Control Algorithm for Interactive Groupware Applications, IEEE ICPADS, 2004, pp. 429–436. [29] W. Lu, Y. Qin, X. Liu, M. Huang, L. Zhou, X. Jiang, Enriching the semantics of variational geometric constraint data with ontology, Comput.–Aided Des. 63 (6) (2015) 72–85. [30] W. Lu, Y. Qin, Q. Qi, W. Zeng, Y. Zhong, X. Liu, X. Jiang, Selecting a semantic similarity measure for concepts in two different CAD model data ontologies, Adv. Eng. Inf. 30 (3) (2016) 449–466. [31] A. Lumsdaine, D. Gregor, B. Hendrickson, J. Berry, Challenges in parallel graph processing, Parallel Processing Letters 17 (01) 5–20. [32] X. Lv, F. He, W. Cai, Y. Cheng, A string-wise CRDT algorithm for smart and large-scale collaborative editing systems, Adv. Eng. Inf. 33 (3) (2017) 397–409. [33] X. Lv, F. He, W. Cai, Y. Cheng, Supporting selective undo of string-wise operations for collaborative editing systems, Future Gener. Comput. Syst. 82 (2018) 41–62. [34] X. Lv, F. He, X. Yan, Y. Wu, Y. Cheng, Integrating selective undo of featurebased modeling operations for real-time collaborative CAD systems, Future Gener. Comput. Syst. (2019) http://dx.doi.org/10.1016/j.future.2019.05.021. [35] M.D. Mechaoui, N. Guetmi, A. Imine, MiCa: Lightweight and mobile collaboration across a collaborative editing service in the cloud, Peer-To-Peer Netw. Appl. 9 (6) (2016) 1242–1269. [36] A. Munir, S. Ranka, A. Gordon-Ross, High-performance energy-efficient multicore embedded computing, IEEE Trans. Parallel Distrib. Syst. 23 (4) (2012) 684–700.

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330 [37] A. Ng, C. Sun, Operational transformation for real-time synchronization of shared workspace in cloud storage, in: Proceedings of the 19th International Conference on Supporting Group Work, ACM, 2016, pp. 61–70. [38] B. Ni, F. He, Y. Pan, Z. Yuan, Using shapes correlation for active contour segmentation of uterine fibroid ultrasound images in computer-aided therapy, Appl. Math.-A J. Chin. Univ. 31 (1) (2016) 37–52. [39] D.A. Nichols, P. Curtis, M. Dixon, J. Lamping, High-latency, High-latency, low-bandwidth windowing in the jupiter collaboration system, in: Proceedings of the 8th Annual ACM Symposium on User Interface and Software Technology, ACM, 1995, pp. 1–120. [40] G. Oster, P. Urso, P. Molli, A. Imine, Imine, data consistency for p2p collaborative editing, in: Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, ACM, 2006, pp. 259–268. [41] Y. Pan, F. He, H. Yu, A correlative denoising autoencoder to model social influence for top-n recommender system, Front. Comput. Sci. (2018) in press. [42] Y. Pan, F. He, H. Yu, A novel enhanced collaborative autoencoder with knowledge distillation for top-n recommender systems, Neurocomputing 332 (2019) 137–148. [43] A. Prakash, M.J. Knister, A framework for undoing actions in collaborative systems, ACM Trans. Comput.-Human Interact. 1 (4) (1994) 295–330. [44] N. Preguica, J.M. Marques, M. Shapiro, M. Letia, A commutative replicated data type for cooperative editing, in: Proceedings of the 29th International Conference on Distributed Computing Systems, IEEE, 395–403. [45] Y.R. Qu, V.K. Prasanna, High-performance and dynamically updatable packet classification engine on fpga, IEEE Trans. Parallel Distrib. Syst. 27 (1) (2016) 197–209. [46] A. Randolph, H. Boucheneb, A. Imine, A. Quintero, On synthesizing a consistent operational transformation approach, IEEE Trans. Comput. 64 (4) (2015) 1074–1089. [47] M. Ressel, D. Nitsche-Ruhland, R. Gunzenhauser, An. integrating, An integrating, transformation-oriented approach to concurrency control and undo in group editors, in: Proceedings of the 1996 ACM Conference on Computer Supported Cooperative Work, ACM, 1996, pp. 58–67. [48] H. Roh, M. Jeon, J. Kim, J. Lee, Replicated abstract data types: Building blocks for collaborative applications, J. Parallel Distrib. Comput. 71 (3) (2011) 354–368. [49] H. Roh, J. Kim, J. Lee, S. Maeng, Optimistic operations for replicated abstract data types, Tech. rep. Technical Report CS-TR-2009-318, KAIST (2009). [50] Y. Saito, M. Shapiro, Optimistic replication, ACM Comput. Surv. 37 (1) (2005) 42–81. [51] B. Shao, D. Li, N. Gu, An algorithm for selective undo of any operation in collaborative applications, in: Proceedings of the 16th ACM International Conference on Supporting Group Work, ACM, 2010, pp. 131–140. [52] M. Shapiro, N. Preguica, C. Baquero, M. Zawirski, A comprehensive study of convergent and commutative replicated data types, Tech. rep. Technical Report inria-00555588, INRIA (2011). [53] H. Shen, C. Sun, Flexible notification for collaborative systems, in: Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work, ACM, 2002, pp. 77–86. [54] M. Singh, V.K. Prasanna, A hierarchical model for distributed collaborative computation in wireless sensor networks, International Journal of Foundations of Computer Science 15 (03) 485–506. [55] P. Stenstrom, M. Brorsson, F. Dahlgren, H. Grahn, M. Dubois, Boosting the performance of shared memory multiprocessors, Computer 30 (7) (1997) 63–70. [56] C. Sun, Undo any operation at any time in group editors, in: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, ACM, 2000, pp. 191–200. [57] C. Sun, Undo as concurrent inverse in group editors, ACM Trans. Comput.-Human Interact. 9 (4) (2002) 309–361. [58] C. Sun, Dependency-conflict detection in real-time collaborative 3d design systems, in: Proceedings of the 2013 Conference on Computer Supported Cooperative Work, ACM, 2013, pp. 715–728. [59] J. Sun, F. He, Y. Chen, X. Chen, A multiple template approach for robust tracking of fast motion target, Appl. Math.-A J. Chin. Univ. 31 (2) (2016) 177–197. [60] C. Sun, X. Jia, Y. Zhang, Y. Yang, D. Chen, Achieving. convergence, causality. preservation, Achieving convergence causality preservation and intention preservation in real-time cooperative editing systems, ACM Trans. Comput.-Human Interact. 5 (1) (1998) 63–108. [61] D. Sun, C. Sun, Context-based operational transformation in distributed collaborative editing systems, IEEE Trans. Parallel Distrib. Syst. 20 (10) (2009) 1454–1470.

329

[62] C. Sun, H. Wen, H. Fan, Operational transformation for orthogonal conflict resolution in real-time collaborative 2d editing systems, in: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, ACM, 2012, pp. 1391–1400. [63] C. Sun, S. Xia, D. Sun, D. Chen, H. Shen, W. Cai, Transparent adaptation of single-user applications for multi-user real-time collaboration, ACM Trans. Comput.-Human Interact. 13 (4) (2006) 531–582. [64] C. Sun, Y. Xu, A. Agustina, Exhaustive search of puzzles in operational transformation, in: Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, ACM, 2014, pp. 519–529. [65] A.S. Tanenbaum, M. Van Steen, Distributed Systems: Principles and Paradigms, Prentice-Hall, 2007. [66] T. Van Luong, N. Melab, E. Talbi, Gpu computing for parallel local search metaheuristic algorithms, IEEE Trans. Comput. 62 (1) (2013) 173–185. [67] N. Vidot, M. Cart, J. Ferri, M. Suleiman, Suleiman, copies convergence in a distributed real-time collaborative environment, in: Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, ACM, 2000, pp. 171–180. [68] S. Weiss, P. Urso, P. Molli, Wooki: a p2p wiki-based collaborative writing tool, in: Proceedings of the International Conference on Web Information Systems Engineering, Springer, 2007, pp. 503–512. [69] S. Weiss, P. Urso, P. Molli, Logoot: a scalable optimistic replication algorithm for collaborative editing on p2p networks, in: Proceedings of the 29th International Conference on Distributed Computing Systems, IEEE, 2009, pp. 404–412. [70] S. Weiss, P. Urso, P. Molli, Logoot-undo: Distributed collaborative editing system on p2p networks, IEEE Trans. Parallel Distrib. Syst. 21 (8) (2010) 1162–1174. [71] M. Wlotzka, V. Heuveline, Energy-efficient multigrid smoothers and grid transfer operators on multi-core and GPU clusters, J. Parallel Distrib. Comput. 100 (2017) 181–192. [72] Y. Wu, F. He, D. Zhang, X. Li, Service-oriented feature-based data exchange for cloud-based design and manufacturing, IEEE Trans. Serv. Comput. 11 (2) (2018) 341–353. [73] S. Xia, D. Sun, C. Sun, D. Chen, H. Shen, Leveraging single-user applications for multi-user collaboration: the coword approach, in: Proceedings of the ACM 2004 Conference on Computer Supported Cooperative Work, ACM, 2004, pp. 162–171. [74] G. Xie, R. Li, K. Li, Heterogeneity–driven end–to–end synchronized scheduling for precedence constrained tasks and messages on networked embedded systems, J. Parallel Distrib. Comput. 83 (2015) 1–12. [75] G. Xie, G. Zeng, J. Jiang, C. Fan, R. Li, K. Li, Energy Management for Multiple Real-time Workflows on Cyber-Physical Cloud Systems, Future Generation Computer Systems http://dx.doi.org/10.1016/j.future.2017.05.033. [76] G. Xie, G. Zeng, X. Xiao, R. Li, K. Li, Energy-efficient scheduling algorithms for real–time parallel applications on heterogeneous distributed embedded systems, IEEE Trans. Parallel Distrib. Syst. 28 (12) (2017) 3426–3442. [77] Y. Xu, C. Sun, Conditions and patterns for achieving convergence in ot-based co-editors, IEEE Trans. Parallel Distrib. Syst. 27 (3) (2016) 695–709. [78] X. Yan, F. He, Y. Chen, A novel hardware/software partitioning method based on position disturbed particle swarm optimization with invasive weed optimization, J. Comput. Sci. Tech. 32 (2) (2017) 340–355. [79] X. Yan, F. He, N. Hou, H. Ai, An efficient particle swarm optimization for large scale hardware/software codesign system, Int. J. Coop. Inf. Syst. 27 (1) (2018) 1741001. [80] W. Yu, Supporting string-wise operations and selective undo for peer-topeer group editing, in: Proceedings of the 18th International Conference on Supporting Group Work, ACM, 2014, pp. 226–237. [81] W. Yu, L. Andr, C. Ignat, A crdt supporting selective undo for collaborative text editing, in: Proceedings of the IFIP International Conference on Distributed Applications and Interoperable Systems, Springer, 2015, pp. 193–206. [82] H. Yu, F. He, Y. Pan, A novel region-based active contour model via local patch similarity measure for image segmentation, Multimedia Tools Appl. 77 (18) (2018) 24097–24119. [83] H. Yu, F. He, Y. Pan, A novel segmentation model for medical images with intensity inhomogeneity based on adaptive perturbation, Multimedia Tools Appl. 78 (9) (2019) 11779–11798. [84] S. Zhang, F. He, W. Ren, J. Yao, Joint learning of image detail and transmission map for single image dehazing, Vis. Comput. (2019) http: //dx.doi.org/10.1007/s00371-018-1612-9. [85] S. Zhao, D. Li, H. Gu, B. Shao, N. Gu, An approach to sharing legacy tv/arcade games for real-time collaboration, in: 29th IEEE International Conference on Distributed Computing Systems, IEEE, 2009, pp. 165–172. [86] Y. Zhou, F. He, N. Hou, Y. Qiu, Parallel ant colony optimization on multi-core simd cpus, Future Gener. Comput. Syst. 79 (2) (2018) 473–487. [87] Y. Zhou, F. He, Y. Qiu, Dynamic strategy based parallel ant colony optimization on gpus for tsps, Sci. China Inf. Sci. 60 (6) (2017) 068102.

330

X. Lv, F. He, W. Cai et al. / Journal of Parallel and Distributed Computing 132 (2019) 310–330 Lv Xiao born in 1983, PH.D., Lecturer. Her research interests include Computer Supported Collaborative Work (CSCW), collaborative CAD.

Cai Wei-Wei born in 1988, PH.D. candidate. His research interests include Computer Supported Collaborative Work (CSCW), collaborative CAD.

He Fa-Zhi born in 1968, PH.D., professor. His research interests include collaborative computing, CAD graphics and images.

Cheng Yuan born in 1983, PH.D., Lecturer. Her research interests include Computer Supported Collaborative Work (CSCW), collaborative CAD.