- Email: [email protected]
Parallel forecasting of community-wide information spread with assimilation of social network data
ScienceDirect ProcediaScienceDirect Computer Science 00 (2018) 000–000
Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
www.elsevier.com/locate/procedia
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 00 (2018) 000–000
Procedia Computer Science 136 (2018) 228–235
7th International Young Scientist Conference on Computational Science 7th International Young Scientist Conference on Computational Science
Parallel forecasting of community-wide information spread with assimilation of social network data spread with Parallel forecasting of community-wide information assimilation of social * Oksana Severiukhina , Sergey Kesarev, Maxnetwork Petrov anddata Klavdiya Bochenina ITMO *University, 197101, 49 Kronverksky St Petersburg, Oksana Severiukhina , Sergey Kesarev, Maxpr.,Petrov andRussia Klavdiya Bochenina ITMO University, 197101, 49 Kronverksky pr., St Petersburg, Russia
Abstract Abstract Nowadays, social networks have become one of the main sources of information. There are many factors affecting the information spreading. On the one hand, we must take into account the features of post and information sources, on the other hand, it is necessary Nowadays, social havetobecome of the main of information. manyand factors affecting the information to understand hownetworks users react posts. one Moreover, socialsources networks (as well as There some are natural technical systems) allow for spreading. Ondata the one hand, must takeofinto account the features of providing post and information sources, on the other hand,the it issimulation. necessary collecting the about the we actual state a dynamical process, thus the input data for assimilation within to users react to posts. Moreover, networksthe (as dissemination well as some of natural and technical allowboth for In understand this paper, how we propose a multi-agent approach social for predicting information taking systems) into account collecting the data data and about the actual state a dynamical process, providing input data forcurrent assimilation within the simulation. retrospective information aboutofthe states of the agentsthus within a socialthe network at the moment. During simulation In this paper, we receives propose network a multi-agent approach predicting information into account both process, the model state data in formfor of batches andthe can dissemination modify internalofparameters for taking more accurate prediction. retrospective data and information about allows the states of the agents withinata social network at the current During simulation The parallel realization of this approach to perform simulation a reasonable time. The resultsmoment. show that the accuracy of process, themay model receives network state data form of batches and can modifyfrom internal parameters more accurate prediction. prediction be significantly improved usinginadditional information collected an online socialfor network. The parallel realization of this approach allows to perform simulation at a reasonable time. The results show that the accuracy of prediction may be significantly using © 2018 The Authors. Publishedimproved by Elsevier B.V.additional information collected from an online social network. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) © 2018 The Authors. Published by Elsevier B.V. © 2018 The Authors. Published by Elsevier Peer-review under responsibility of the B.V. scientific committee of the 7th International Young Scientist Conference on This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) This is an open access article under the CC (http://creativecommons.org/licenses/by-nc-nd/3.0/) Computational Science. Peer-review under responsibility of the scientificBY-NC-ND committee oflicense the 7th International Young Scientist Conference on Computational Science. Peer-review under responsibility of the scientific committee of the 7th International Young Scientist Conference on Keywords: social Science. network; parallel algorithm; forecasting; information spread Computational Keywords: social network; parallel algorithm; forecasting; information spread
1. Introduction
1. Introduction The growth of social networks gave rise to a variety of predictive models of dynamical processes in cyberspace. It can be used for a various range of purposes from market predicting in business to rumor controlling and opinion The growth for of social networks Most gave rise to models a varietyrepresent of predictive models dynamical processes in cyberspace. It can monitoring a government. of the a system as aofsingle-layer or multiplex complex network be used for a various range of purposes from market predicting in business to rumor controlling and opinion monitoring for a government. Most of the models represent a system as a single-layer or multiplex complex network *
Corresponding author. E-mail address: [email protected]
*
Corresponding author. E-mail address: [email protected] 1877-0509 © 2018 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review the scientific committee 1877-0509 ©under 2018responsibility The Authors. of Published by Elsevier B.V.of the 7th International Young Scientist Conference on Computational Science. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the 7th International Young Scientist Conference on Computational Science. 1877-0509 © 2018 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the 7th International Young Scientist Conference on Computational Science. 10.1016/j.procs.2018.08.260
2
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
229
while studying the relation between topology of this network and its functional properties (e.g. conductivity). Moreover, the huge size of resulting networks often leads to the development of parallel implementation of models. The typical pattern of simulation using the model of information spread on a complex network is, given initial conditions (a set of activated nodes), to transmit information messages iteratively to the neighbours of activated nodes. To use a model, one needs to specify the probability of ‘infection’, that is, the probability of reaction of a given node (user) to a given information message. For models, when one is interested to simulate the dynamics of cascade on a given topology, fixed probabilities are usually used as they allow to compare the effectiveness of different networks in terms of information dissemination. However, for the case of super-spreaders (communities) which we explore in this study, this approach does not work due to a typically small depth of cascade (subscribers of a community tend to share information directly from the source, not from their friends who are also subscribers). The second problem for modeling of community-wide information spread is that different information messages generated by the source can have different ‘virality’ (the level of potential impact on the audience) according to the content of the message. In the models of individual reactions, this problem is usually solved by training an additional model which outputs a probability of reaction for the pair (user, information message type) according to the observed history of his or her reactions. Although this approach allows to support heterogeneity of nodes, it is of restricted applicability for the case of a multi-community landscape when new types of communities (and new types of information messages) may be frequently added into consideration. The remainder of this paper is organized as follows. Section 2 contains some background information and related works. In Section 3 we present method for parallel forecasting of community-wide information spread and describe its communication with the crawler. Section 4 presents dataset description and results of simulations. Conclusions and plans for future works can be found in Section 5. 2. Related works In general, models studying the processes of information dissemination in social networks can be divided into two main types. The first type of models aims to explain processes in social networks. The second type of models is based on the first type and is devoted to forecasting information spreading. For a more realistic simulation of the processes, it is necessary to explore social networks and identify the key features of the entities and their interactions. In the article [1], there is a list of points for realistic information spreading model with behavioral analysis: the popularity of the source, a strength of relations among users, a content of the information, personal interests, privacy preferences. However, these parameters have different effects and models can both include them in consideration and omit depending on the purposes of modeling. In addition, an important role is played by the network structure, which determines the relationship between individual elements of the network by channels, communities, users and etc. Ferrara in the article [2] identified three main features that characterize the structure of online social networks: “Small World” effect, scale-free degree distributions and, finally, the emergence of a community structure. Popular models for graph generation like Erdos-Renyi random graphs, Watt-Strogatz and Barabasi-Albert model cannot reproduce all the properties. This is the reason for using data about the real network structure. The conductivity of links between users may influence on information spreading, also the users who have more common friends have a greater weight for the dissemination of information [3]. The prediction of the dissemination of information is important from the point of view of the government, companies and individuals. It helps to control rumor or opinion spreading, predict marketing or other actions. Authors of [4] distinguish three types of predictive models: • independent cascade model (ICM); • linear threshold model (LTM); • the game theory model. In ICM inactive node can be activated by the active node with some predefined probability. In the second type of models, communication between nodes provides a cumulative effect on state. An active node can try to activate inactive nodes several times to pass a predefined threshold for changing a state of inactive nodes. However, it should be noted that the task of forecasting is closely related to the task of maximizing the influence or dissemination of information. For example, for the case of scalable influence maximization authors use independent cascade [5] or
230
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
3
linear threshold [6] models. In the game theory model, there are some specific restrictions and strategies for nodes. For example, model [7] is aimed to explain how human factors impact competitive information dissemination. It should be noted that nodes in social networks have heterogenous nature and some individual characteristics may influence processes in networks. The agent’s curiosity may improve the diffusion of information in traditional spreading models [8]. Research [9] takes into account heterogeneity of stateful agents. Post content in a social network can include a lot of information, such as text, hashtags, pictures, geocoordinates, attachments. In addition, posts can be divided by type of information: news, contests, voting, shares from other community, etc. Nicola Barbieri in work [10] proposes topic-aware independent cascade and topic-aware linear threshold models with different distribution topics and strength of influence of nodes on each other depending on the topic. The huge size of real networks became a reason for a parallel realisation of a model. It can be applied on different steps of simulation from generative models (e.g. parallel Chung-Lu model [11]) to models of information spread (e.g. parallel SIR [12]). Recently, more and more began to appear research devoted to predicting the dissemination of information in social networks. For example in the article [13], Quan et al. study repost prediction based on temporal behaviour patterns of users. Authors of [14] use a machine learning approach with the passive-aggressive algorithm for automatically prediction share behavior in tweeter. However, the widest application of forecasting with data assimilation found in application for modeling the weather and climate change. Thus, the majority of existing algorithms for information prediction use artificial networks, homogeneous agents and messages, fixed probabilities of information transfer. However, features of the individual behavior of users affect the whole process of information spreading (from micro to more complex macro level). In this article, we propose parallel forecasting of community-wide information spreading model which based on the composition of models and consider heterogeneous of entities. In addition, the model takes the data constantly in parts from the crawler and adjusts the parameters. This solves the problem of operational forecasting. 3. Method 3.1. Description of the general scheme For the scenario of operational forecasting of the effect of an information message, one needs to combine pretraining (i.e. preliminary identification of parameters of model of information spread) with real-time model tuning according to a periodically observed actual state of the process. From one side, this approach allows to reduce the impact of characteristics of previously unobserved types of information messages on the results of prediction. From the other side, it naturally helps to decrease an error of forecast by data assimilation. In this paper, we consider the
Fig. 1. General scheme of the method.
4
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
231
example of such approach for the prediction of reaction on a series of information messages in large thematic communities of an online social network. The general scheme of the method is shown in Fig. 1. Social networks are a graph where the vertices are individual users, public pages, communities, meetings. There are two types of relations as edges: friendship and subscription. An example of friendship is the relationship between users, an example of a subscription is the user's membership in the community. Social networks contain a large amount of information including information about the posts and the reaction of users to them. For data collection we used the crawler system [15]. The system allows us to maintain effective collection data from largest Russian online social network vk.com (further referred as VK). We developed special scenarios [16] for data collection from different VK communities. The first scenario collects posts of a given group and filter it out by time, only posts with a lifetime less than a month will be saved to the storage in a JSON file format. When posts will be collected, for each post crawler generates tasks, which collect likes, shares (reposts), comments of the post and then store it to the files. These scenarios are shown below in Fig. 2.
Fig. 2. Сrawler scenarios: collect posts, likes, comments, shares and followers of the community.
Unfortunately, another problem occurs here: there is no information about follower subscription time and like time. A solution here is monitoring feature of the crawler, which can have the following interface (Fig. 3): one just needs to provide interval and duration of monitoring.
Fig. 3. Crawler scenario for monitoring subscriptions of followers of the community.
Thus, the crawler makes it possible to obtain data of two types: historical data over the past time intervals and network data at the current time. However, further application of the data in the model requires additional processing of the received files. Historical data is needed to adjust the input parameters of the model, such as: • a network of subscriptions and friendship; • generation frequency and types of information messages for communities; • user online activity in the network (active/inactive); • types of users and the probability of user reactions (like, share, comment) to various types of received messages. As mentioned above, the information from the crawler comes in the form of batches. The batches are a set of files in JSON format that contain a lot of additional information, for example, the exact share time. However, in our
232
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
5
model, data is assimilated with a certain interval, so we process the data of batches and get aggregated characteristics on the news: the number of reactions from different groups of users at the time of collection of information. The new data in the model are added as sequential batches, which are processed and added to a specific folder in the model. The social network data at the current time allows to configure the internal parameters of the model, such as: • number of reactions to news (likes, shares, comments); • number of news’ views; • virality coefficient. For studying information spreading in cyberspace, we expand our model, which is described in detail in previous papers [17], [18]. Model consists of three internal models: a model of IM’s generation, a model of activity and model of reaction. The first model determines a creation of new messages. Model of activity defines the active or inactive status of each agent. Reaction model is responsible for user’s reaction to incoming messages: inaction, approval (like), the generation of a text message (comment), participation in the dissemination of information (share). The presented model allows to simulate various processes and configure the model parameters: changing characteristics of internal models, settings of parameters using processed data from social networks. Parallel implementation of the proposed approach allows to simulate large-scale information spreading processes in networks. The main simulation scheme then is as follows: the model of information cascades in a thematic community is pretrained by the history of reactions of users on a series of posts on a community's page. When a new post appears, its value of virality is calculated as an average of previous observations, and the model is launched with a set of baseline parameters. At the same time, periodic monitoring of the real influence of a given post is performed with the use of web crawler. These data, in turn, serve as an input data for the model, allowing to tune the internal parameters thus leading to the reducing of forecasting error. When simulation ends, accumulated data are used to update baseline values of parameters. 3.2. Prediction algorithm To implement the model in the scheme described above, our previous model [17] was modified. It has a parallel implementation for the ability to model large-scale networks using a cluster or supercomputer resources. It was implemented with C++ language and MPI standard for message interchange. Algorithm 1 shows the basic steps in the model when obtaining new data from the crawler. Algorithm 1. Data assimilation by model
Input: initialization parameters, sequence of batches 1:
initializeParameters()
2:
while !existNewBatch:
3:
runModel()
4:
b = getNewBatch()
5:
saveForecatingResults()
6:
returnToBatchStep(b)
7:
compareReaction(b)
8:
modificationMessageParameters(b)
9:
goto: step 2
During the first step, the initialization of initial parameters of internal models takes place. In the generation model, the types of posts and laws for their publication (a frequency, a distribution publication per day and week) for communities of different types are given. The model of user activity determines the probability of their being online at different intervals of the day and can differ for different types of users (for example, early birds, night people). The reaction model is based on probabilistic laws and has reaction parameters depending on the type of user, news for different types of reactions. All the above parameters are based on historical crawler data.
6
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
233
After that, a check is made for the presence of a batch about the current state of the system (step 2). In the absence of a batch, the model is launched (step 3). Calculations occur until the moment when a new batch appears. In addition, this scheme allows to read new data from the batch at predetermined intervals. After receiving the new batch (step 4), you must save the current simulation results a few steps forward (step 5) and roll back the changes at the time of the batch (step 6). Step 7 is responsible for comparing the simulation results and the batch data. There are probably 2 cases: • if the observed informational impact is larger in online social network that in the model, then it is necessary to add some of the potential viewers for news and increase the counters for news; • if there is an opposite situation, then, on the contrary, it is necessary to reduce the counters on the news (users lose reactions but remain in the viewers). The next step 8 is to change the message parameters. Each message has vector of virality, which has three components 𝑣𝑣 = (𝑣𝑣𝑙𝑙 , 𝑣𝑣𝑠𝑠 , 𝑣𝑣𝑐𝑐 ). Where 𝑣𝑣𝑙𝑙 , 𝑣𝑣𝑟𝑟 , 𝑣𝑣𝑐𝑐 is virality for like, share and comment reaction. Each of them can increase or decrease the probability of user reaction to the news and is recalculated after applying of batch according to the following formula: (1)
𝑣𝑣𝑟𝑟,𝑘𝑘+1 = 𝑣𝑣𝑟𝑟,𝑘𝑘 ∙ (1 − (𝑐𝑐𝑟𝑟,𝑚𝑚 − 𝑐𝑐𝑟𝑟,𝑏𝑏 )⁄𝑐𝑐𝑟𝑟,𝑏𝑏 )
where 𝑟𝑟 = {𝑙𝑙, 𝑠𝑠, 𝑐𝑐} is one of possible reactions, 𝑘𝑘 is a batch number, 𝑐𝑐𝑟𝑟,𝑚𝑚 is number of reactions in the model, 𝑐𝑐𝑟𝑟,𝑏𝑏 is number of reaction in the current batch. After applying the changes to the batch, the modeling process is run again until the network state updates with the new batch (step 9). 4. Experimental study 4.1. Dataset description The research of information forecasting processes for news in social networks was held on the example of users of VK. The network was formed on the basis of a public community dedicated to news and its subscribers. Historical data were collected by the crawler for 2 months. All data are represented as a set of JSON strings. During this time, 1587 posts were published in the community (1479 regular news posts and 22 advertising posts). Thus, we can distinguish two basic types of posts, the distribution of reactions is very different from the type of information (Fig. 4a, y axis values in the logarithmic scale). Thus, each post will have a different impact on the audience's reaction. This value we determine by the virality coefficient. Its normalized mean value is equal to 1. However, there is also heterogeneity among users in the network under consideration. So, among all subscribers (1 871 022 users) during two months, only 491 016 was active. The Figure 4b shows the proportion of reactions of different users according to which it is possible to determine the roles of users in the community. For this figure, user data with more than 100 activities were used. It is worth noting the following feature of the VK network: if the user decided to share the information, then not only the share is added to the reaction counters, but also the like. However, users can remove this like reaction. a
b
Fig. 4. (a) Reaction of community’s followers to different types of posts; (b) a proportion of reactions.
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
234
7
4.2. Simulation results In our model we used the Master-Slave pattern for communication between processes. One master process provides data synchronization and generation of news; slave processes perform calculations. In this example, we simulated forecast for 1 day in advance after publication time. Simulation time for the forecast was 20 minutes for the version with two slave processes and 8 minutes for the version with 7 slave processes. Thus, current implementation allows you to significantly reduce the calculation time with increasing processes number. Nevertheless, the efficiency of the parallel version will be studied in detail and improved in future studies. This is strongly necessary for forecasts for longer periods of time, for example, a month in advance. a
b
Fig. 5. Dynamics of the number of reactions: (a) likes; (b) shares.
For modeling, several news items of different types were taken from the community in question. Crawler has sent batches with information about the current state in the network, after processing the data were available in the model and the parameters were updated. In this study, we consider only one day after the publication of the post. The simulation results for several types of reactions are shown in Figure 5. On these Figures there are 2 types of data, the dots represent real data from the network, obtained from the batch with information. Lines are forecasts, each line begins after the application of a batch. As we see from the graphs, the forecast of informational impact of the post can be larger or smaller than the actual values. However, the presented model can take into account these deviations and change the number of responses and the dynamics of the change in the number of reactions. In order to estimate the forecast error, we use the metric mean absolute error (MAPE). This metric is considered at the points of receiving the batch data, the number of points is determined by the time until the end of modeling this news since the publication. In this case, the news is Fig. 6. Error dynamics (MAPE). modeled within one day. As you can see from Figure 6, the forecast error decreases as the batch number increases. 5. Conclusion and future works In this paper, we propose an approach for parallel forecasting of community-wide information spread with assimilation of social network data. The approach is based on discrete multi-agent modeling and a combination of internal models (generation, reaction and activity models) and allows to adjust the model parameters for more accurate results. The experimental part of the study was aimed to estimate the extent to which the error of forecasting may be reduced with proposed scheme of data assimilation. The parallel simulation program was combined with crawler of
8
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
235
OSN vk.com (namely, crawler periodically sends the data to a model, and the simulation program recalculates its parameters using the updated parameters). The quality of resulting forecasts (with and without data assimilation) was compared for a large community. Parallel implementation of the proposed algorithm allows you to perform calculations on large networks in reasonable time. Thus, we can get a forecast for some time ahead and refine it as new data appears that were sent by the crawler. Among the future works, two main areas of research can be distinguished. The first is devoted to more accurate forecasting and taking into account the reactions of other users, as a separate part of the model. The second part will be devoted to modeling the longitudinal evolution of user states in terms of their level of involvement in interaction with the community. Acknowledgements This research was supported by The Russian Scientific Foundation, Agreement #14–21–00137–П (02.05.2017). References [1] B. Sayin and S. Şahin, “A Novel Approach to Information Spreading Models for Social Networks,” DATA Anal. 2017 Sixth Int. Conf. Data Anal. III., 2017. [2] S. Cuomo, G. Magnete, V. Orabona, and M. Chinnici, “Topological Features of Online Social Networks,” Commun. Appl. Ind. Math., pp. 1– 14, 2012. [3] C. Ou, X. Jin, Y. Wang, and X. Cheng, “Modelling heterogeneous information spreading abilities of social network ties,” Simul. Model. Pract. Theory, vol. 75, pp. 67–76, 2017. [4] M. Li, X. Wang, K. Gao, and S. Zhang, “A Survey on Information Diffusion in Online Social Networks: Models and Methods,” Information, vol. 8, no. 4, p. 118, Sep. 2017. [5] K. Jung, W. Heo, and W. Chen, “IRIE: Scalable and robust influence maximization in social networks,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2012, pp. 918–923. [6] W. Chen, Y. Yuan, and L. Zhang, “Scalable influence maximization in social networks under the linear threshold model,” in Data Mining (ICDM), 2010 IEEE 10th International Conference on, 2010, pp. 88–97. [7] Q. Sun and Z. Yao, “Evolutionary game analysis of competitive information dissemination on social networks: An agent-based computational approach,” Math. Probl. Eng., vol. 2015, pp. 1–12, Jun. 2015. [8] D. A. Vega-Oliveros, L. Berton, F. Vazquez, and F. A. Rodrigues, “The Impact of Social Curiosity on Information Spreading on Networks,” 2017. [9] Z. Q. Zhu, C. J. Liu, J. L. Wu, J. Xu, and B. Liu, “The Influence of Human Heterogeneity to Information Spreading,” J. Stat. Phys., vol. 154, no. 6, pp. 1569–1577, 2014. [10] N. Barbieri, F. Bonchi, and G. Manco, “Topic-aware social influence propagation models,” Knowl. Inf. Syst., vol. 37, no. 3, pp. 555–584, 2013. [11] M. Alam and M. Khan, “Parallel Algorithms for Generating Random Networks with Given Degree Sequences,” Int. J. Parallel Program., vol. 45, no. 1, pp. 109–127, 2017. [12] A. Bhatele et al., “Massively parallel simulations of spread of infectious diseases over realistic social networks,” in Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017, 2017, pp. 689–694. [13] Y. Quan, Y. Jia, B. Zhou, W. Han, and S. Li, “Repost prediction incorporating time-sensitive mutual influence in social networks,” Journal of Computational Science, Apr-2018. [14] S. Petrovic, M. Osborne, and V. Lavrenko, “Rt to win! predicting message propagation in twitter,” Proc. Fifth Int. Conf. Weblogs Soc. Media - ICWSM ’11, pp. 586–589, 2011. [15] N. Butakov, M. Petrov, and A. Radice, “Multitenant Approach to Crawling of Online Social Networks,” in Procedia Computer Science, 2016, vol. 101, pp. 115–124. [16] N. Butakov, M. Petrov, K. Mukhina, D. Nasonov, and S. Kovalchuk, “Unified domain-specific language for collecting and processing data of social media,” J. Intell. Inf. Syst., pp. 1–26, May 2018. [17] O. Severiukhina, K. Bochenina, S. Kesarev, and A. Boukhanovsky, “Parallel data-driven modeling of information spread in social networks,” Procedia Comput. Sci., vol. 10860, pp. 247–259, Jun. 2018. [18] S. Kesarev, O. Severiukhina, and K. Bochenina, “Parallel simulation of community-wide information spreading in online social networks,” Commun. Comput. Inf. Sci., 2018.
Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
www.elsevier.com/locate/procedia
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 00 (2018) 000–000
Procedia Computer Science 136 (2018) 228–235
7th International Young Scientist Conference on Computational Science 7th International Young Scientist Conference on Computational Science
Parallel forecasting of community-wide information spread with assimilation of social network data spread with Parallel forecasting of community-wide information assimilation of social * Oksana Severiukhina , Sergey Kesarev, Maxnetwork Petrov anddata Klavdiya Bochenina ITMO *University, 197101, 49 Kronverksky St Petersburg, Oksana Severiukhina , Sergey Kesarev, Maxpr.,Petrov andRussia Klavdiya Bochenina ITMO University, 197101, 49 Kronverksky pr., St Petersburg, Russia
Abstract Abstract Nowadays, social networks have become one of the main sources of information. There are many factors affecting the information spreading. On the one hand, we must take into account the features of post and information sources, on the other hand, it is necessary Nowadays, social havetobecome of the main of information. manyand factors affecting the information to understand hownetworks users react posts. one Moreover, socialsources networks (as well as There some are natural technical systems) allow for spreading. Ondata the one hand, must takeofinto account the features of providing post and information sources, on the other hand,the it issimulation. necessary collecting the about the we actual state a dynamical process, thus the input data for assimilation within to users react to posts. Moreover, networksthe (as dissemination well as some of natural and technical allowboth for In understand this paper, how we propose a multi-agent approach social for predicting information taking systems) into account collecting the data data and about the actual state a dynamical process, providing input data forcurrent assimilation within the simulation. retrospective information aboutofthe states of the agentsthus within a socialthe network at the moment. During simulation In this paper, we receives propose network a multi-agent approach predicting information into account both process, the model state data in formfor of batches andthe can dissemination modify internalofparameters for taking more accurate prediction. retrospective data and information about allows the states of the agents withinata social network at the current During simulation The parallel realization of this approach to perform simulation a reasonable time. The resultsmoment. show that the accuracy of process, themay model receives network state data form of batches and can modifyfrom internal parameters more accurate prediction. prediction be significantly improved usinginadditional information collected an online socialfor network. The parallel realization of this approach allows to perform simulation at a reasonable time. The results show that the accuracy of prediction may be significantly using © 2018 The Authors. Publishedimproved by Elsevier B.V.additional information collected from an online social network. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) © 2018 The Authors. Published by Elsevier B.V. © 2018 The Authors. Published by Elsevier Peer-review under responsibility of the B.V. scientific committee of the 7th International Young Scientist Conference on This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) This is an open access article under the CC (http://creativecommons.org/licenses/by-nc-nd/3.0/) Computational Science. Peer-review under responsibility of the scientificBY-NC-ND committee oflicense the 7th International Young Scientist Conference on Computational Science. Peer-review under responsibility of the scientific committee of the 7th International Young Scientist Conference on Keywords: social Science. network; parallel algorithm; forecasting; information spread Computational Keywords: social network; parallel algorithm; forecasting; information spread
1. Introduction
1. Introduction The growth of social networks gave rise to a variety of predictive models of dynamical processes in cyberspace. It can be used for a various range of purposes from market predicting in business to rumor controlling and opinion The growth for of social networks Most gave rise to models a varietyrepresent of predictive models dynamical processes in cyberspace. It can monitoring a government. of the a system as aofsingle-layer or multiplex complex network be used for a various range of purposes from market predicting in business to rumor controlling and opinion monitoring for a government. Most of the models represent a system as a single-layer or multiplex complex network *
Corresponding author. E-mail address: [email protected]
*
Corresponding author. E-mail address: [email protected] 1877-0509 © 2018 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review the scientific committee 1877-0509 ©under 2018responsibility The Authors. of Published by Elsevier B.V.of the 7th International Young Scientist Conference on Computational Science. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the 7th International Young Scientist Conference on Computational Science. 1877-0509 © 2018 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/3.0/) Peer-review under responsibility of the scientific committee of the 7th International Young Scientist Conference on Computational Science. 10.1016/j.procs.2018.08.260
2
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
229
while studying the relation between topology of this network and its functional properties (e.g. conductivity). Moreover, the huge size of resulting networks often leads to the development of parallel implementation of models. The typical pattern of simulation using the model of information spread on a complex network is, given initial conditions (a set of activated nodes), to transmit information messages iteratively to the neighbours of activated nodes. To use a model, one needs to specify the probability of ‘infection’, that is, the probability of reaction of a given node (user) to a given information message. For models, when one is interested to simulate the dynamics of cascade on a given topology, fixed probabilities are usually used as they allow to compare the effectiveness of different networks in terms of information dissemination. However, for the case of super-spreaders (communities) which we explore in this study, this approach does not work due to a typically small depth of cascade (subscribers of a community tend to share information directly from the source, not from their friends who are also subscribers). The second problem for modeling of community-wide information spread is that different information messages generated by the source can have different ‘virality’ (the level of potential impact on the audience) according to the content of the message. In the models of individual reactions, this problem is usually solved by training an additional model which outputs a probability of reaction for the pair (user, information message type) according to the observed history of his or her reactions. Although this approach allows to support heterogeneity of nodes, it is of restricted applicability for the case of a multi-community landscape when new types of communities (and new types of information messages) may be frequently added into consideration. The remainder of this paper is organized as follows. Section 2 contains some background information and related works. In Section 3 we present method for parallel forecasting of community-wide information spread and describe its communication with the crawler. Section 4 presents dataset description and results of simulations. Conclusions and plans for future works can be found in Section 5. 2. Related works In general, models studying the processes of information dissemination in social networks can be divided into two main types. The first type of models aims to explain processes in social networks. The second type of models is based on the first type and is devoted to forecasting information spreading. For a more realistic simulation of the processes, it is necessary to explore social networks and identify the key features of the entities and their interactions. In the article [1], there is a list of points for realistic information spreading model with behavioral analysis: the popularity of the source, a strength of relations among users, a content of the information, personal interests, privacy preferences. However, these parameters have different effects and models can both include them in consideration and omit depending on the purposes of modeling. In addition, an important role is played by the network structure, which determines the relationship between individual elements of the network by channels, communities, users and etc. Ferrara in the article [2] identified three main features that characterize the structure of online social networks: “Small World” effect, scale-free degree distributions and, finally, the emergence of a community structure. Popular models for graph generation like Erdos-Renyi random graphs, Watt-Strogatz and Barabasi-Albert model cannot reproduce all the properties. This is the reason for using data about the real network structure. The conductivity of links between users may influence on information spreading, also the users who have more common friends have a greater weight for the dissemination of information [3]. The prediction of the dissemination of information is important from the point of view of the government, companies and individuals. It helps to control rumor or opinion spreading, predict marketing or other actions. Authors of [4] distinguish three types of predictive models: • independent cascade model (ICM); • linear threshold model (LTM); • the game theory model. In ICM inactive node can be activated by the active node with some predefined probability. In the second type of models, communication between nodes provides a cumulative effect on state. An active node can try to activate inactive nodes several times to pass a predefined threshold for changing a state of inactive nodes. However, it should be noted that the task of forecasting is closely related to the task of maximizing the influence or dissemination of information. For example, for the case of scalable influence maximization authors use independent cascade [5] or
230
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
3
linear threshold [6] models. In the game theory model, there are some specific restrictions and strategies for nodes. For example, model [7] is aimed to explain how human factors impact competitive information dissemination. It should be noted that nodes in social networks have heterogenous nature and some individual characteristics may influence processes in networks. The agent’s curiosity may improve the diffusion of information in traditional spreading models [8]. Research [9] takes into account heterogeneity of stateful agents. Post content in a social network can include a lot of information, such as text, hashtags, pictures, geocoordinates, attachments. In addition, posts can be divided by type of information: news, contests, voting, shares from other community, etc. Nicola Barbieri in work [10] proposes topic-aware independent cascade and topic-aware linear threshold models with different distribution topics and strength of influence of nodes on each other depending on the topic. The huge size of real networks became a reason for a parallel realisation of a model. It can be applied on different steps of simulation from generative models (e.g. parallel Chung-Lu model [11]) to models of information spread (e.g. parallel SIR [12]). Recently, more and more began to appear research devoted to predicting the dissemination of information in social networks. For example in the article [13], Quan et al. study repost prediction based on temporal behaviour patterns of users. Authors of [14] use a machine learning approach with the passive-aggressive algorithm for automatically prediction share behavior in tweeter. However, the widest application of forecasting with data assimilation found in application for modeling the weather and climate change. Thus, the majority of existing algorithms for information prediction use artificial networks, homogeneous agents and messages, fixed probabilities of information transfer. However, features of the individual behavior of users affect the whole process of information spreading (from micro to more complex macro level). In this article, we propose parallel forecasting of community-wide information spreading model which based on the composition of models and consider heterogeneous of entities. In addition, the model takes the data constantly in parts from the crawler and adjusts the parameters. This solves the problem of operational forecasting. 3. Method 3.1. Description of the general scheme For the scenario of operational forecasting of the effect of an information message, one needs to combine pretraining (i.e. preliminary identification of parameters of model of information spread) with real-time model tuning according to a periodically observed actual state of the process. From one side, this approach allows to reduce the impact of characteristics of previously unobserved types of information messages on the results of prediction. From the other side, it naturally helps to decrease an error of forecast by data assimilation. In this paper, we consider the
Fig. 1. General scheme of the method.
4
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
231
example of such approach for the prediction of reaction on a series of information messages in large thematic communities of an online social network. The general scheme of the method is shown in Fig. 1. Social networks are a graph where the vertices are individual users, public pages, communities, meetings. There are two types of relations as edges: friendship and subscription. An example of friendship is the relationship between users, an example of a subscription is the user's membership in the community. Social networks contain a large amount of information including information about the posts and the reaction of users to them. For data collection we used the crawler system [15]. The system allows us to maintain effective collection data from largest Russian online social network vk.com (further referred as VK). We developed special scenarios [16] for data collection from different VK communities. The first scenario collects posts of a given group and filter it out by time, only posts with a lifetime less than a month will be saved to the storage in a JSON file format. When posts will be collected, for each post crawler generates tasks, which collect likes, shares (reposts), comments of the post and then store it to the files. These scenarios are shown below in Fig. 2.
Fig. 2. Сrawler scenarios: collect posts, likes, comments, shares and followers of the community.
Unfortunately, another problem occurs here: there is no information about follower subscription time and like time. A solution here is monitoring feature of the crawler, which can have the following interface (Fig. 3): one just needs to provide interval and duration of monitoring.
Fig. 3. Crawler scenario for monitoring subscriptions of followers of the community.
Thus, the crawler makes it possible to obtain data of two types: historical data over the past time intervals and network data at the current time. However, further application of the data in the model requires additional processing of the received files. Historical data is needed to adjust the input parameters of the model, such as: • a network of subscriptions and friendship; • generation frequency and types of information messages for communities; • user online activity in the network (active/inactive); • types of users and the probability of user reactions (like, share, comment) to various types of received messages. As mentioned above, the information from the crawler comes in the form of batches. The batches are a set of files in JSON format that contain a lot of additional information, for example, the exact share time. However, in our
232
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
5
model, data is assimilated with a certain interval, so we process the data of batches and get aggregated characteristics on the news: the number of reactions from different groups of users at the time of collection of information. The new data in the model are added as sequential batches, which are processed and added to a specific folder in the model. The social network data at the current time allows to configure the internal parameters of the model, such as: • number of reactions to news (likes, shares, comments); • number of news’ views; • virality coefficient. For studying information spreading in cyberspace, we expand our model, which is described in detail in previous papers [17], [18]. Model consists of three internal models: a model of IM’s generation, a model of activity and model of reaction. The first model determines a creation of new messages. Model of activity defines the active or inactive status of each agent. Reaction model is responsible for user’s reaction to incoming messages: inaction, approval (like), the generation of a text message (comment), participation in the dissemination of information (share). The presented model allows to simulate various processes and configure the model parameters: changing characteristics of internal models, settings of parameters using processed data from social networks. Parallel implementation of the proposed approach allows to simulate large-scale information spreading processes in networks. The main simulation scheme then is as follows: the model of information cascades in a thematic community is pretrained by the history of reactions of users on a series of posts on a community's page. When a new post appears, its value of virality is calculated as an average of previous observations, and the model is launched with a set of baseline parameters. At the same time, periodic monitoring of the real influence of a given post is performed with the use of web crawler. These data, in turn, serve as an input data for the model, allowing to tune the internal parameters thus leading to the reducing of forecasting error. When simulation ends, accumulated data are used to update baseline values of parameters. 3.2. Prediction algorithm To implement the model in the scheme described above, our previous model [17] was modified. It has a parallel implementation for the ability to model large-scale networks using a cluster or supercomputer resources. It was implemented with C++ language and MPI standard for message interchange. Algorithm 1 shows the basic steps in the model when obtaining new data from the crawler. Algorithm 1. Data assimilation by model
Input: initialization parameters, sequence of batches 1:
initializeParameters()
2:
while !existNewBatch:
3:
runModel()
4:
b = getNewBatch()
5:
saveForecatingResults()
6:
returnToBatchStep(b)
7:
compareReaction(b)
8:
modificationMessageParameters(b)
9:
goto: step 2
During the first step, the initialization of initial parameters of internal models takes place. In the generation model, the types of posts and laws for their publication (a frequency, a distribution publication per day and week) for communities of different types are given. The model of user activity determines the probability of their being online at different intervals of the day and can differ for different types of users (for example, early birds, night people). The reaction model is based on probabilistic laws and has reaction parameters depending on the type of user, news for different types of reactions. All the above parameters are based on historical crawler data.
6
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
233
After that, a check is made for the presence of a batch about the current state of the system (step 2). In the absence of a batch, the model is launched (step 3). Calculations occur until the moment when a new batch appears. In addition, this scheme allows to read new data from the batch at predetermined intervals. After receiving the new batch (step 4), you must save the current simulation results a few steps forward (step 5) and roll back the changes at the time of the batch (step 6). Step 7 is responsible for comparing the simulation results and the batch data. There are probably 2 cases: • if the observed informational impact is larger in online social network that in the model, then it is necessary to add some of the potential viewers for news and increase the counters for news; • if there is an opposite situation, then, on the contrary, it is necessary to reduce the counters on the news (users lose reactions but remain in the viewers). The next step 8 is to change the message parameters. Each message has vector of virality, which has three components 𝑣𝑣 = (𝑣𝑣𝑙𝑙 , 𝑣𝑣𝑠𝑠 , 𝑣𝑣𝑐𝑐 ). Where 𝑣𝑣𝑙𝑙 , 𝑣𝑣𝑟𝑟 , 𝑣𝑣𝑐𝑐 is virality for like, share and comment reaction. Each of them can increase or decrease the probability of user reaction to the news and is recalculated after applying of batch according to the following formula: (1)
𝑣𝑣𝑟𝑟,𝑘𝑘+1 = 𝑣𝑣𝑟𝑟,𝑘𝑘 ∙ (1 − (𝑐𝑐𝑟𝑟,𝑚𝑚 − 𝑐𝑐𝑟𝑟,𝑏𝑏 )⁄𝑐𝑐𝑟𝑟,𝑏𝑏 )
where 𝑟𝑟 = {𝑙𝑙, 𝑠𝑠, 𝑐𝑐} is one of possible reactions, 𝑘𝑘 is a batch number, 𝑐𝑐𝑟𝑟,𝑚𝑚 is number of reactions in the model, 𝑐𝑐𝑟𝑟,𝑏𝑏 is number of reaction in the current batch. After applying the changes to the batch, the modeling process is run again until the network state updates with the new batch (step 9). 4. Experimental study 4.1. Dataset description The research of information forecasting processes for news in social networks was held on the example of users of VK. The network was formed on the basis of a public community dedicated to news and its subscribers. Historical data were collected by the crawler for 2 months. All data are represented as a set of JSON strings. During this time, 1587 posts were published in the community (1479 regular news posts and 22 advertising posts). Thus, we can distinguish two basic types of posts, the distribution of reactions is very different from the type of information (Fig. 4a, y axis values in the logarithmic scale). Thus, each post will have a different impact on the audience's reaction. This value we determine by the virality coefficient. Its normalized mean value is equal to 1. However, there is also heterogeneity among users in the network under consideration. So, among all subscribers (1 871 022 users) during two months, only 491 016 was active. The Figure 4b shows the proportion of reactions of different users according to which it is possible to determine the roles of users in the community. For this figure, user data with more than 100 activities were used. It is worth noting the following feature of the VK network: if the user decided to share the information, then not only the share is added to the reaction counters, but also the like. However, users can remove this like reaction. a
b
Fig. 4. (a) Reaction of community’s followers to different types of posts; (b) a proportion of reactions.
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
234
7
4.2. Simulation results In our model we used the Master-Slave pattern for communication between processes. One master process provides data synchronization and generation of news; slave processes perform calculations. In this example, we simulated forecast for 1 day in advance after publication time. Simulation time for the forecast was 20 minutes for the version with two slave processes and 8 minutes for the version with 7 slave processes. Thus, current implementation allows you to significantly reduce the calculation time with increasing processes number. Nevertheless, the efficiency of the parallel version will be studied in detail and improved in future studies. This is strongly necessary for forecasts for longer periods of time, for example, a month in advance. a
b
Fig. 5. Dynamics of the number of reactions: (a) likes; (b) shares.
For modeling, several news items of different types were taken from the community in question. Crawler has sent batches with information about the current state in the network, after processing the data were available in the model and the parameters were updated. In this study, we consider only one day after the publication of the post. The simulation results for several types of reactions are shown in Figure 5. On these Figures there are 2 types of data, the dots represent real data from the network, obtained from the batch with information. Lines are forecasts, each line begins after the application of a batch. As we see from the graphs, the forecast of informational impact of the post can be larger or smaller than the actual values. However, the presented model can take into account these deviations and change the number of responses and the dynamics of the change in the number of reactions. In order to estimate the forecast error, we use the metric mean absolute error (MAPE). This metric is considered at the points of receiving the batch data, the number of points is determined by the time until the end of modeling this news since the publication. In this case, the news is Fig. 6. Error dynamics (MAPE). modeled within one day. As you can see from Figure 6, the forecast error decreases as the batch number increases. 5. Conclusion and future works In this paper, we propose an approach for parallel forecasting of community-wide information spread with assimilation of social network data. The approach is based on discrete multi-agent modeling and a combination of internal models (generation, reaction and activity models) and allows to adjust the model parameters for more accurate results. The experimental part of the study was aimed to estimate the extent to which the error of forecasting may be reduced with proposed scheme of data assimilation. The parallel simulation program was combined with crawler of
8
Oksana Severiukhina et al. / Procedia Computer Science 136 (2018) 228–235 Author name / Procedia Computer Science 00 (2018) 000–000
235
OSN vk.com (namely, crawler periodically sends the data to a model, and the simulation program recalculates its parameters using the updated parameters). The quality of resulting forecasts (with and without data assimilation) was compared for a large community. Parallel implementation of the proposed algorithm allows you to perform calculations on large networks in reasonable time. Thus, we can get a forecast for some time ahead and refine it as new data appears that were sent by the crawler. Among the future works, two main areas of research can be distinguished. The first is devoted to more accurate forecasting and taking into account the reactions of other users, as a separate part of the model. The second part will be devoted to modeling the longitudinal evolution of user states in terms of their level of involvement in interaction with the community. Acknowledgements This research was supported by The Russian Scientific Foundation, Agreement #14–21–00137–П (02.05.2017). References [1] B. Sayin and S. Şahin, “A Novel Approach to Information Spreading Models for Social Networks,” DATA Anal. 2017 Sixth Int. Conf. Data Anal. III., 2017. [2] S. Cuomo, G. Magnete, V. Orabona, and M. Chinnici, “Topological Features of Online Social Networks,” Commun. Appl. Ind. Math., pp. 1– 14, 2012. [3] C. Ou, X. Jin, Y. Wang, and X. Cheng, “Modelling heterogeneous information spreading abilities of social network ties,” Simul. Model. Pract. Theory, vol. 75, pp. 67–76, 2017. [4] M. Li, X. Wang, K. Gao, and S. Zhang, “A Survey on Information Diffusion in Online Social Networks: Models and Methods,” Information, vol. 8, no. 4, p. 118, Sep. 2017. [5] K. Jung, W. Heo, and W. Chen, “IRIE: Scalable and robust influence maximization in social networks,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2012, pp. 918–923. [6] W. Chen, Y. Yuan, and L. Zhang, “Scalable influence maximization in social networks under the linear threshold model,” in Data Mining (ICDM), 2010 IEEE 10th International Conference on, 2010, pp. 88–97. [7] Q. Sun and Z. Yao, “Evolutionary game analysis of competitive information dissemination on social networks: An agent-based computational approach,” Math. Probl. Eng., vol. 2015, pp. 1–12, Jun. 2015. [8] D. A. Vega-Oliveros, L. Berton, F. Vazquez, and F. A. Rodrigues, “The Impact of Social Curiosity on Information Spreading on Networks,” 2017. [9] Z. Q. Zhu, C. J. Liu, J. L. Wu, J. Xu, and B. Liu, “The Influence of Human Heterogeneity to Information Spreading,” J. Stat. Phys., vol. 154, no. 6, pp. 1569–1577, 2014. [10] N. Barbieri, F. Bonchi, and G. Manco, “Topic-aware social influence propagation models,” Knowl. Inf. Syst., vol. 37, no. 3, pp. 555–584, 2013. [11] M. Alam and M. Khan, “Parallel Algorithms for Generating Random Networks with Given Degree Sequences,” Int. J. Parallel Program., vol. 45, no. 1, pp. 109–127, 2017. [12] A. Bhatele et al., “Massively parallel simulations of spread of infectious diseases over realistic social networks,” in Proceedings - 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2017, 2017, pp. 689–694. [13] Y. Quan, Y. Jia, B. Zhou, W. Han, and S. Li, “Repost prediction incorporating time-sensitive mutual influence in social networks,” Journal of Computational Science, Apr-2018. [14] S. Petrovic, M. Osborne, and V. Lavrenko, “Rt to win! predicting message propagation in twitter,” Proc. Fifth Int. Conf. Weblogs Soc. Media - ICWSM ’11, pp. 586–589, 2011. [15] N. Butakov, M. Petrov, and A. Radice, “Multitenant Approach to Crawling of Online Social Networks,” in Procedia Computer Science, 2016, vol. 101, pp. 115–124. [16] N. Butakov, M. Petrov, K. Mukhina, D. Nasonov, and S. Kovalchuk, “Unified domain-specific language for collecting and processing data of social media,” J. Intell. Inf. Syst., pp. 1–26, May 2018. [17] O. Severiukhina, K. Bochenina, S. Kesarev, and A. Boukhanovsky, “Parallel data-driven modeling of information spread in social networks,” Procedia Comput. Sci., vol. 10860, pp. 247–259, Jun. 2018. [18] S. Kesarev, O. Severiukhina, and K. Bochenina, “Parallel simulation of community-wide information spreading in online social networks,” Commun. Comput. Inf. Sci., 2018.