- Email: [email protected]
Urban events prediction via convolutional neural networks and Instagram data
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science (2019) 000–000 Procedia Computer Science 15600 (2019) 176–184
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
8th 8th International International Young Young Scientist Scientist Conference Conference on on Computational Computational Science Science
Urban Urban events events prediction prediction via via convolutional convolutional neural neural networks networks and and Instagram Instagram data data a,∗ a a Ksenia Ksenia D. D. Mukhina Mukhinaa,∗,, Alexander Alexander A. A. Visheratin Visheratina ,, Denis Denis Nasonov Nasonova a ITMO a ITMO
University, Saint Petersburg, Russia, 199034 University, Saint Petersburg, Russia, 199034
Abstract Abstract In today’s world, it is crucial to be proactive and be prepared for events which are not happening yet. Thus, there is no surprise In today’s world, it is crucial to be proactive and be prepared for events which are not happening yet. Thus, there is no surprise that in the field of social media analysis the research agenda has moved from the development of event detection methods to a that in the field of social media analysis the research agenda has moved from the development of event detection methods to a brand new area – event prediction models. This research field is extremely important for all sorts of applications, from natural brand new area – event prediction models. This research field is extremely important for all sorts of applications, from natural disasters preparation and criminal activity prevention to urban management and development of smart cities. However, even the disasters preparation and criminal activity prevention to urban management and development of smart cities. However, even the leading models have an important disadvantage: they are based on prior knowledge about events being expected. So forecasting leading models have an important disadvantage: they are based on prior knowledge about events being expected. So forecasting systems based on such models are heavily limited by a list of events that can be predicted and all events of other types will be out systems based on such models are heavily limited by a list of events that can be predicted and all events of other types will be out of systems’ scope. In this work, we try to address this issue and propose a deep learning model, which is able to predict an area of of systems’ scope. In this work, we try to address this issue and propose a deep learning model, which is able to predict an area of the future event in the urban environment. This model is able to predict the future state of the city – a level of users activity in the the future event in the urban environment. This model is able to predict the future state of the city – a level of users activity in the location-based social network Instagram – with the average deviation from the ground truth of 1%, and achieves 69% recall when location-based social network Instagram – with the average deviation from the ground truth of 1%, and achieves 69% recall when solving the events prediction problem. solving the events prediction problem. c 2019 The Authors. Authors. Published by by Elsevier Ltd. © c 2019 The Authors. Published Published by Elsevier Ltd. BY-NC-ND license license https://creativecommons.org/licenses/by-nc-nd/4.0/) (https://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND This is an open access article under the CC BY-NC-ND license https://creativecommons.org/licenses/by-nc-nd/4.0/) responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Peer-review under responsibility Peer-review under responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Science. Science. Science. Keywords: convolution neural network; deep learning; Instagram; social network; event prediction Keywords: convolution neural network; deep learning; Instagram; social network; event prediction
1. Introduction 1. Introduction We live in times where new technologies is a necessary part of every day life. We surrounded by recommendation We live in times where new technologies is a necessary part of every day life. We surrounded by recommendation systems of various kinds, starting from music recommendations to events suggestions. Thus, there was no surprise systems of various kinds, starting from music recommendations to events suggestions. Thus, there was no surprise when data from social media became a valuable part of such systems. For example, data from social media is actively when data from social media became a valuable part of such systems. For example, data from social media is actively used in event detection systems. Prediction of events in the city is even more complex task because it has all the same used in event detection systems. Prediction of events in the city is even more complex task because it has all the same problems as events detection (noise, data representation) combined with problems of the prediction domain (model problems as events detection (noise, data representation) combined with problems of the prediction domain (model selection, under/over-fitting). selection, under/over-fitting). ∗ ∗
Corresponding author. Corresponding author. E-mail address: [email protected] E-mail address: [email protected]
c 2019 The Authors. Published by Elsevier Ltd. 1877-0509 c 2019 The Authors. Published by Elsevier Ltd. 1877-0509 This is an open access under the CC BY-NC-ND license https://creativecommons.org/licenses/by-nc-nd/4.0/) 1877-0509 © Thearticle Authors. Published by Elsevier Ltd. This is an open2019 access article under the CC BY-NC-ND license https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review underaccess responsibility of the scientific committee oflicense the 8th (https://creativecommons.org/licenses/by-nc-nd/4.0/) International Young Scientist Conference on Computational Science. This is an open article under the CC BY-NC-ND Peer-review under responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Science. Peer-review under responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Science. 10.1016/j.procs.2019.08.193
2
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
177
There are two main approaches for events prediction in social networks. The first one is centered around the user activity patterns. User posts are considered as active responses to changes in the environment, and users themselves act as sensors [11]. Users friends list and previous check-ins are used to identify the next locations with the highest probability. The standard approach is to consider the user-location interaction as a link [12] since it allows to use various techniques starting from network analysis to machine learning for link prediction. Nevertheless, this approach limits forecasting systems to a set of particular users only and prediction of the state for some area becomes a hard task. Another way to forecast events or social network state is to predict changes in the popularity of a specific topic. These methods rely on text analysis and feature extraction. Although such techniques allow predicting an emergence of some trend in advance accurately, they have several serious drawbacks. First, some trended topics may not be related to an actual event; for example, it could be an active discussion or a flash mob. That is why all detected trends should be additionally checked, and list of observing topics should be specified. Secondly, such narrowing of monitoring domain could result in the absence of some events in the list of topics. This could happen in case when events of a particular type are unexpected in the studied area, i.e. street concerts, protests, etc. Thereby, there are two different issues, which should be taken into account for event prediction using social networks data. To avoid disadvantages of user- and content-based approaches, we propose a method for the city state prediction, which can determine the next state independently from a predefined message context or list of monitored users. Our method relies on a historical information of spatial activity in a social network. We take into consideration only spatiotemporal distribution of posts; usage of only geographical and temporal data allows to forecast changes in users behavior on a city scale. The method is based on creating dense grids for every hour of the available data, filling cells of these grids with aggregated statistics about users activity, and using grids for several previous hours to predict the next state. As a result, the model predicts the activity level in each cell for the next hour. Representation of a region state as a spatial grid opens doors to modern approaches for prediction such as machine learning and deep learning techniques [2]. In this work, we use Convolutional Neural Network model where city state is interpreted as a two-dimensional matrix. Convolutional neural networks (CNN) is well-known approach proven to be efficient for image classification and pattern recognition; they are also actively used for event detection and next frame prediction for video data. In this approach image is represented as a two-dimensional matrix where each cell corresponds to a pixel of the image and video is a sequence of such matrix. Since in our case, we also represent data as a matrix, and our primary goal is to predict the next state based on spatiotemporal patterns of historical data that is why we used CNN model for city state prediction. For a thorough evaluation of the developed approaches, we collected a large dataset of Instagram posts in the New York City area from 2010 to 2018. The resulting dataset contains more than 62 million posts in 114,000 locations. This allowed us to create baseline adaptive geogrids for the full year and investigate the efficiency of our solution in several experiments devoted to determining a precision rate and a recall rate. Experimental results show that urban state forecasting model achieves the recall of 69.4%.
2. Related works The modern studies dedicated to social network analysis moved one step further from the event detection to the event prediction [6]. In [1], the authors pointed out that in the modern era it is crucial to be proactive and be prepared for events which do not happen yet. This is extremely important for all sort of emergency events: natural disasters or criminal activity. However, information about cultural and fun events is also relevant for the municipal services and should be predicted as well. The most common way to predict events is to associate the event with a specified list of keywords or topics; these methods are focused on forecasting upcoming trends. For instance, in [18] Zhao et al. used the list of 124 predefined keywords related to flu for event prediction using Twitter. Usage of such wide list allowed to predict flu outbreaks with 94% of precision and the recall was 84% on the state level where precision is the percentage of the predictions which are actual events and recall is fraction of actual events which was predicted by model. Despite the fact that proposed approach achieves better results on the higher scale such as region and country level, other experiments showed that on the city level the recall value exceeds 50% only for three countries out of eight. Their own method based on Hidden
178
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
3
Markov Model and presented in [17] demonstrates better results with 69% of recall for flu outbreaks detection in New York and 70% of recall for civil unrest events in Mexico. In [13], the list of 72 terms was used to forecast flu outbreaks by Twitter stream. Data from Centers for Disease Control and Prevention was used to evaluate the results. Similar to Zhaos work authors used one week time interval for the prediction of flu since this time frame used in control dataset. The event considered as predicted if there was an alert within seven days prior to the event. The true positive rate for developed method is 0.55. Thus, the one of major issues in the event prediction is to predict all occurring events which is hard for term-based methods since they require excessive list of keywords for each of possible event. In [5] Huang et al. proposed a novel method to ensure three essential issues in event prediction: (a) the earliest possible prediction of the event, (b) observing dynamic of the event and (c) extraction of coherent words to the event. In presented approach authors interpreted the event as a topic which is actively discussed in some microblog. They developed emerging topic tracking algorithm (ETT) for monitoring of emerging, growing and fading states of the topic. This algorithm utilizes spatial and temporal features to monitor the topic emergence and its following life-cycle. Authors used Sina Weibo microblog service to demonstrate the effectiveness of their methods. Despite experiments showed 77% accuracy on the topic emergence, this method detects the event only with its beginning. Thus, the prediction task was remained unsolved by authors. The ensemble of models for crime events prediction using Foursquare data was presented in [10]. Rumi et al. split the day into three-hours periods and tried to predict the occurrence of crime event in the next time interval. The historical, geographic, demographic and dynamic features were used in the model and results showed that the use of dynamic features leads to improvement of the accuracy of predictions. Authors chose New York City and Brisbane as their target cities for experiments and achieved up to 99% accuracy for Brisbane and 77% for New York. These results can be explained by the fact that Brisbane is a safer city than New York. During the experiments, authors compared ensemble with the performance of each model separately. It was shown that Random Forest and Neural Network perform with the comparable results to the ensemble of models. However, the presented method relies not only on Foursquare data but also on historical data about crimes and demographic data which are not available for every region. In addition to that, this approach is focused on type of events crime events and does not cover other possible events. Thus, prediction of city state requires novel methods which are capable of forecasting activity in geographic area in general. The applicability of Deep Neural Network (DNN) to event prediction was also demonstrated in [15]. Authors used linear Support Vector Machine, Decision Tree, Random Forest and DNN for vehicle crash prediction based on demographic and weather data. It was shown that DNN achieves the highest accuracy and usage of the spatial features such as road network connectivity improves this result. In [4] it was shown that event detection methods can be successfully used for prediction because the modern methods of event retrieval identify event up to two days before the news agencies, the earthquake in Latin America was taken as example and even the worst performing method detected event 2.04 days ahead of news reports. The CNN architecture was developed by Farajidavar et al. [3] for early event detection in Twitter. The main goal of this research was to identify type of occurring event correctly. To achieve that authors used dataset labeled in 7 major event categories: food, environment, social, sport, crime, cultural, and transport events. The model performed with 81% accuracy on the event extraction. However, this method is based on text features, so it cannot be extrapolated on the geographic areas with any preferable language but English. Moreover, presented model is limited by aforementioned event classes so it might fail in case of mixed-classes events or events which are hard to range in the certain class, for example, local fairs. Existing methods for event forecasting for the most part focus on predicting popularity of topic represented by a list of keywords. To achieve the satisfactory results, such methods require long lists of terms (in some cases more than 100) associated with topic. If these terms are commonly used words, it becomes hard to identify a true location of the event. Thus, novel methods utilizing temporal and spatial features of data becoming more relevant. Approaches based on deep learning even though they were designed for specific types of events show promising results and in this work, we use convolutional neural network to predict events of diverse types for following hour.
4
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
179
Fig. 1. Data processing scheme
3. Dataset For the event prediction, we used the data from Instagram. We selected a time period from 1 January 2017 till 18 April 2018 and gathered data from 108846 locations. We used the first 12 months for model training, and all the data for 2018 was used for validation and model evaluation. Since we train our model on the full year, we can be sure that seasonal trends are taken into consideration. We select five-hours timeframe as input data for prediction, this size of time widow was chosen in order to balance between daily dynamics and current trend. Analysis of the impact of different time windows on the model performance is discussed in next section. In addition to that, during aforementioned period the number of Instagram profiles have grown approximately on 300 millions of users, and by the end of June, the number of active users has reached more than 1 billion profiles. So, by achieving satisfying results with our model, we can expect sufficient results from our model with following Instagram expansion. The basic scheme of data processing is presented in Figure 1. Due to the computational limitations, we cut the region of interest for prediction to the city center, so the final rectangle was bounded by coordinates [-74.058257, 73.856257, 40.63484, 40.82884], the covered area is shown on Figure 1, left. We used only timestamp and coordinates of each post. On the first step, we constructed a dense grid with spacing equal to 0.0005 and placed all posts for each hour separately. Thus, we obtained the initial grid with activity statistics for each hour of each day for the whole time period where cell value corresponds to a number of photos posted during that specific hour. In our approach, we use five previous hours to predict the following one (Figure 1, middle). So, we construct three-dimensional tensor with shape [5x404x388] where the first dimension represents previous time slices and second and third dimensions correspond to latitude and longitude, respectively. These tensors are used as an input data for our prediction model and similar one-dimensional tensor for the following hour (Figure 1, right) represents a desirable result of prediction. Thus, our model is capable of forecasting future users activity starting from the sixth hour in the obtained data. 4. Method description As it was previously explained, Convolutional Neural Networks is a promising and natural approach for this kind of a task. In our previous study, we have shown that CNN model can predict an urban area state with up to 99% accuracy on the average [8]. Thus, in this work, we decided to use the network architecture, which performed best. The deep network architecture is presented in Figure 2; this architecture is based on six three-dimensional convolutional layers with ELU as an activation function. As can be seen from the schema, input data convolves by all three dimensions: width, height, and depth that represents time. To obtain the final grid coverage the same area as in the input data, we switch the kernel size to k = (1,3,3) when time dimension achieved size 1 and keep this value till the last layer. The stride and padding value were similar for each layer and equal s = 1 and p = (0,1,1) respectively. The L1 loss was used in the model with ADADELTA optimizer algorithm [16] with starting learning rate lr = 0.5. The choice of these
180
Ksenia D. et al.Computer / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et Mukhina al. / Procedia Science 00 (2019) 000–000
5
Fig. 2. Proposed CNN architecture
parameters was determined by evaluation of model performance, and it was explained in details in our previous work [8]. For the performance evaluation, we used the same function, which is an average deviation percentage from the ground truth dense grid. Since dense grid represents the actual activity level in the city, it has some low and extremely high activity areas with sharp transitions. For accurate event detection, it is important to correctly predict areas with high and medium activity areas since they may be event indicators. Nevertheless, it is also important to predict areas with zero activity to reduce the number of assumed event candidates that should be checked. This following function was designed to balance between accurate prediction of high and low activity areas:
ev =
N 1 | yi j − yˆ i j | · 100%, N i, j yˆ i j
(1)
where yi j is a prediction matrix element, and yˆ i j is the matching cell of ground truth dense grid. In contrast to our previous work, we focused on the city center and used the dense grid with half as large step size. First, since city center usually attracts more people and more events take place in center comparing with districts, with smaller step size we can achieve better accuracy since we will get smooth transitions between high and low activity areas. Second, step size equal to 0.0005 matches the size of the initial statistical grid for event detection method. Thus, we will be able to compare the result obtained by event detection with our predictions. Since we changed the grid and step size, the network was trained for this area separately. In addition to that, we conducted a series of experiments to identify an optimal time window length; for each launch, network was trained for 100 epochs. We used the workstation equipped by NVIDIA Tesla P100, and the model was implemented with the help of PyTorch framework [9]. The results of experiments for parameter adjustment are demonstrated on Figure 3. Time window size was varied between 3, 5, and 7 hours in order to achieve the balance between the current situation in the city and repeating daily dynamics. As can be seen from the plot the average deviation shows almost no improvements during the training for seven hours with rising in error at the end, meantime, for three hours there is a constant upward trend. In the same time, the average deviation for the five-hour window demonstrated strong declining trend and reached its minimum at 87th epoch. Thus, since model performs best with a five-hour time window, we selected this time frame for the following analysis. Finally, the described combination of model parameters with this size of time window allows achieving the best average deviation which is 0.95% for the center of New York City. The result of prediction model represents a dense tensor of activity in the area, where the value in each cell equals to the expected number of posts in certain geographic area. Since the prediction model relies on spatial and temporal data only, it is impossible to forecast the content or type of events. But based on previous activity dynamics, neural network is able to highlight areas with a high probability of some event in the future. Result matrix can play the role of
6
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
181
Fig. 3. Effect of different time-window on CNN performance
statistical grid and be used as an input data for anomalies search. However, resulting tensor consists of float number, meanwhile number of posts is an integer number. So, first, the output data is rounded up to the closest integer. On the last step, predicted dense grid with activity values is used in the event detection pipeline. Thus, we obtain the list of candidates as a result of event prediction algorithm. In next section we compare results obtained by our prediction model with actual events identified by our event detection method. 5. Experiments In this section we analyze the quality of results of event prediction model. For the evaluation of the prediction model, we decided to pick the same two weeks from the previous study dedicated to event detection [14]: active week (12-18 March 2018) and non-active week (19-25 February 2018). This event detection method allows to achieve up to 100% of entertainment and beauty events with 77.1% of precision on average on various categories. Since there is no reliable source of all events in the city (news media usually covers only urgent events; meanwhile concert halls put a schedule on their websites), results of event detection method become the complete list of past events. To compare the results obtained by adaptive geogrids, the historical grids were constructed for the city center based on the same dense grid, which is used in the CNN. We ran anomalies search method on the prediction model results and obtained the list of candidates for these two weeks. In Figure 4, the heatmap for predicted activity matrix is presented. As can be seen from the map even in the city center high and low activity areas differ drastically. Even though in some areas predicted activity level does not precisely match with an actual number of posts, in the majority of cases CNN was able to correctly predict zero activity areas as well as the most active places. Moreover, the average deviation is equaled to 0.95%, i.e. CNN based on three-dimensional convolutional layers ensures 99.05% accuracy for the center of the New York City. Since in this experiment we predict only event candidates it is more important to get an accurate prediction of all detected events than to decrease the false positive rate and predict event only. In Table 1, results of the model are presented, we compared predicted candidates with actual events obtained by adaptive geogrids during experiments in previous section. Table 1. Summary of event prediction results
Active week Non-active week Total
True events
Detected events
%
87 96 183
58 69 127
66.7% 71.8% 69.4%
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
182
7
Fig. 4. Comparison of the predicted level of activity (left) with ground truth data (right)
The event is considered as correctly predicted if there is a respective candidate obtained from the prediction results by adaptive geogrids. Since with different amount of data we get different structure of ConvTree we looked for overlapping between leaves where an actual event was detected and where a candidate was obtained. Recall value is 69.4%, which complies with the best result for recall in this area [17, 18]. In the prediction model we do not have context information about predicted posts, and consequently, we cannot distinct high active area from actual events like we do in the event detection method. Even though the number of candidates highly exceeds the number of actual events (Table 2), prediction model reduces the number of leaves for further analysis as well as geographic area of possible events by 33%. Thus, our model can be successfully used in decision support systems or early-warning systems where it is important to have knowledge of upcoming events in prior. Table 2. Event candidates summary
Event detection Event prediction
Actual events
False candidates
%
183 127
2100 1402
8.7% 9%
It is important to note here that we consider event as predicted if we have a candidate for the exact same hour as it is in event detection algorithms result. Thus, it is possible that some event was correctly predicted not in the beginning of the event but during following hours. For example, concerts last for several hours and exhibitions take places for several months. To overcome this issue, we need to obtain the full list of events with their durations. One of possible way to create such dataset is to use our event detection method and merge the same event from different hours. It can be done by analyzing co-occurrence of the same hashtags in geographically close leaves from different grids. On the other side, in this work we try to forecast events for following hour. In the meantime, some events such as holidays parades or protests are planned in advance, so they can be predicted by analyzing year patterns or context of messages. It was shown that in social media first language is prevalent [7], thus, one of the important challenge will be to develop methods for different languages. However, the main goal of this work was to predict as many events of various nature as possible so in this method we did not use additional data on purpose.
8
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
183
6. Conclusion In this work we presented a prediction model based on a convolutional neural network. This model allows to predict users activity in the area with 1% average deviation from the ground truth data and achieve 69.4% recall on the event prediction task. Nevertheless, we see two directions to enhance the proposed forecasting method. First, we need to decrease false positive rate and to lower the number of detected candidates. It can be achieved by using attention techniques in the neural network. This improvement would also impact on the performance in term of computation time and memory consuming. Second direction is to try more complicated models for the prediction. In this work, we used only convolutional layers, but combination of recurrent neural network and convolutional neural network approaches may identify more complicated spatiotemporal activity patterns. In addition to that, we are going to use context from data for the prediction because for some cultural event people are very excited and start to actively post several hours prior to event. Nonetheless, event forecasting methods based on the spatiotemporal features of data showed sufficient results for their tasks. The experiments demonstrated that usage of three-dimensional convolutions for event forecasting performs on the same level as the best and most complicated methods in this area. Acknowledgements This research is financially supported by the Russian Science Foundation, Agreement #18-71-00149. References [1] Dencik, L., Hintz, A., Carey, Z., 2018. Prediction, pre-emption and limits to dissent: Social media and big data uses for policing protests in the United Kingdom. New Media and Society 20, 1433–1450. doi:10.1177/1461444817697722. [2] Do, T.H., Nguyen, D.M., Tsiligianni, E., Cornelis, B., Deligiannis, N., 2018. Twitter User Geolocation Using Deep Multiview Learning, in: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 6304– 6308. URL: http://homepages.vub.ac.be/{~}ndeligia/pubs/icassp{_}18{_}tien.pdf, doi:10.1109/ICASSP.2018.8462191, arXiv:1805.04612. [3] Farajidavar, N., Kolozali, S., Barnaghi, P., 2017. A Deep Multi-View Learning Framework for City Event Extraction from Twitter Data Streams URL: http://arxiv.org/abs/1705.09975, arXiv:1705.09975. [4] Hua, T., Chen, F., Zhao, L., Lu, C.T., Ramakrishnan, N., 2016. Automatic targeted-domain spatiotemporal event detection in twitter. volume 20. GeoInformatica. URL: http://dx.doi.org/10.1007/s10707-016-0263-0, doi:10.1007/s10707-016-0263-0. [5] Huang, J., Peng, M., Wang, H., Cao, J., Gao, W., Zhang, X., 2017. A probabilistic method for emerging topic tracking in Microblog stream. World Wide Web 20, 325–350. URL: https://doi.org/10.1007/s11280-016-0390-4, doi:10.1007/s11280-016-0390-4. [6] Kursuncu, U., Gaur, M., Lokala, U., Thirunarayan, K., Sheth, A., Arpinar, I.B., 2018. Predictive Analysis on Twitter: Techniques and Applications , 1–37URL: http://arxiv.org/abs/1806.02377, doi:10.1007/978-3-319-94105-9_4, arXiv:1806.02377. [7] Magdy, A., Mokbel, M.F., Elnikety, S., Nath, S., He, Y., 2014. Mercury: A Memory-Constrained Spatio-temporal Real-time Search on Microblogs, in: Proceedings - International Conference on Data Engineering. URL: https://www.microsoft.com/en-us/research/ wp-content/uploads/2016/02/samehe-icde2014.mercury.pdf. [8] Mukhina, K.D., Visheratin, A.A., Mbogo, G.K., Nasonov, D., 2018. Forecasting of the Urban Area State Using Convolutional Neural Networks. 2018 23rd Conference of Open Innovations Association (FRUCT) , 268–275URL: https://www.theverge.com/2018/6/20/17484420/. [9] Paszke, A., Chanan, G., Lin, Z., Gross, S., Yang, E., Antiga, L., Devito, Z., 2017. Automatic differentiation in PyTorch. Advances in Neural Information Processing Systems 30 . [10] Rumi, S.K., Deng, K., Salim, F.D., 2018. Crime event prediction with dynamic features. EPJ Data Science 7, 43. URL: http://dx.doi. org/10.1140/epjds/s13688-018-0171-7, doi:10.1140/epjds/s13688-018-0171-7. [11] Sakaki, T., Okazaki, M., Matsuo, Y., 2010. Earthquake Shakes Twitter Users: Real-time Event Detection by Social Sensors. Proceedings of the 19th International Conference on World Wide Web doi:10.1145/1772690.1772777, arXiv:0808.0743v3. [12] Scellato, S., Mascolo, C., 2011. Exploiting Place Features in Link Prediction on Location-based Social Networks Categories and Subject Descriptors, in: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1046–1054. URL: http://www1.se.cuhk.edu.hk/{~}seem5010/slides/kdd2011{_}placefriends.pdf, doi:10.1145/2020408.2020575. [13] Shao, M., Li, J., Chen, F., Huang, H., Zhang, S., Chen, X., 2017. An Efficient Approach to Event Detection and Forecasting in Dynamic Multivariate Social Media Networks, in: Proceedings of the 26th International Conference on World Wide Web - WWW ’17. doi:10.1145/ 3038912.3052588. [14] Visheratin, A.A., Mukhina, K.D., Visheratina, A.K., Nasonov, D., Boukhanovsky, A.V., 2018. Multiscale Event Detection Using Convolutional Quadtrees and Adaptive Geogrids, in: Proceedings of the 2Nd ACM SIGSPATIAL Workshop on Analytics for Local Events and News, ACM, New York, NY, USA. pp. 1:1—-1:10. URL: http://doi.acm.org/10.1145/3282866.3282867, doi:10.1145/3282866.3282867.
184
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
9
[15] Yuan, Z., Tamerius, J., 2017. Predicting Traffic Accidents Through Heterogeneous Urban Data: A Case Study. The 6th International Workshop on Urban Computing URL: https://doi.org/10.475/123{_}4. [16] Zeiler, M.D., 2012. ADADELTA: An Adaptive Learning Rate Method. arXiv URL: http://arxiv.org/abs/1212.5701, doi:http: //doi.acm.org.ezproxy.lib.ucf.edu/10.1145/1830483.1830503, arXiv:1212.5701. [17] Zhao, L., Chen, F., Lu, C.T., Ramakrishnan, N., 2015. Spatiotemporal Event Forecasting in Social Media. Proceedings of the 2015 SIAM International Conference on Data Mining , 963–971URL: http://epubs.siam.org/doi/10.1137/1.9781611974010.108, doi:10.1137/ 1.9781611974010.108. [18] Zhao, L., Chen, F., Lu, C.T., Ramakrishnan, N., 2017. Multi-resolution spatial event forecasting in social media. Proceedings IEEE International Conference on Data Mining, ICDM , 689–698URL: http://www.nvc.cs.vt.edu/{~}ctlu/Publication/2016/ ICDM-Proceedings-2016.pdf, doi:10.1109/ICDM.2016.75.
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science (2019) 000–000 Procedia Computer Science 15600 (2019) 176–184
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
8th 8th International International Young Young Scientist Scientist Conference Conference on on Computational Computational Science Science
Urban Urban events events prediction prediction via via convolutional convolutional neural neural networks networks and and Instagram Instagram data data a,∗ a a Ksenia Ksenia D. D. Mukhina Mukhinaa,∗,, Alexander Alexander A. A. Visheratin Visheratina ,, Denis Denis Nasonov Nasonova a ITMO a ITMO
University, Saint Petersburg, Russia, 199034 University, Saint Petersburg, Russia, 199034
Abstract Abstract In today’s world, it is crucial to be proactive and be prepared for events which are not happening yet. Thus, there is no surprise In today’s world, it is crucial to be proactive and be prepared for events which are not happening yet. Thus, there is no surprise that in the field of social media analysis the research agenda has moved from the development of event detection methods to a that in the field of social media analysis the research agenda has moved from the development of event detection methods to a brand new area – event prediction models. This research field is extremely important for all sorts of applications, from natural brand new area – event prediction models. This research field is extremely important for all sorts of applications, from natural disasters preparation and criminal activity prevention to urban management and development of smart cities. However, even the disasters preparation and criminal activity prevention to urban management and development of smart cities. However, even the leading models have an important disadvantage: they are based on prior knowledge about events being expected. So forecasting leading models have an important disadvantage: they are based on prior knowledge about events being expected. So forecasting systems based on such models are heavily limited by a list of events that can be predicted and all events of other types will be out systems based on such models are heavily limited by a list of events that can be predicted and all events of other types will be out of systems’ scope. In this work, we try to address this issue and propose a deep learning model, which is able to predict an area of of systems’ scope. In this work, we try to address this issue and propose a deep learning model, which is able to predict an area of the future event in the urban environment. This model is able to predict the future state of the city – a level of users activity in the the future event in the urban environment. This model is able to predict the future state of the city – a level of users activity in the location-based social network Instagram – with the average deviation from the ground truth of 1%, and achieves 69% recall when location-based social network Instagram – with the average deviation from the ground truth of 1%, and achieves 69% recall when solving the events prediction problem. solving the events prediction problem. c 2019 The Authors. Authors. Published by by Elsevier Ltd. © c 2019 The Authors. Published Published by Elsevier Ltd. BY-NC-ND license license https://creativecommons.org/licenses/by-nc-nd/4.0/) (https://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND This is an open access article under the CC BY-NC-ND license https://creativecommons.org/licenses/by-nc-nd/4.0/) responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Peer-review under responsibility Peer-review under responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Science. Science. Science. Keywords: convolution neural network; deep learning; Instagram; social network; event prediction Keywords: convolution neural network; deep learning; Instagram; social network; event prediction
1. Introduction 1. Introduction We live in times where new technologies is a necessary part of every day life. We surrounded by recommendation We live in times where new technologies is a necessary part of every day life. We surrounded by recommendation systems of various kinds, starting from music recommendations to events suggestions. Thus, there was no surprise systems of various kinds, starting from music recommendations to events suggestions. Thus, there was no surprise when data from social media became a valuable part of such systems. For example, data from social media is actively when data from social media became a valuable part of such systems. For example, data from social media is actively used in event detection systems. Prediction of events in the city is even more complex task because it has all the same used in event detection systems. Prediction of events in the city is even more complex task because it has all the same problems as events detection (noise, data representation) combined with problems of the prediction domain (model problems as events detection (noise, data representation) combined with problems of the prediction domain (model selection, under/over-fitting). selection, under/over-fitting). ∗ ∗
Corresponding author. Corresponding author. E-mail address: [email protected] E-mail address: [email protected]
c 2019 The Authors. Published by Elsevier Ltd. 1877-0509 c 2019 The Authors. Published by Elsevier Ltd. 1877-0509 This is an open access under the CC BY-NC-ND license https://creativecommons.org/licenses/by-nc-nd/4.0/) 1877-0509 © Thearticle Authors. Published by Elsevier Ltd. This is an open2019 access article under the CC BY-NC-ND license https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review underaccess responsibility of the scientific committee oflicense the 8th (https://creativecommons.org/licenses/by-nc-nd/4.0/) International Young Scientist Conference on Computational Science. This is an open article under the CC BY-NC-ND Peer-review under responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Science. Peer-review under responsibility of the scientific committee of the 8th International Young Scientist Conference on Computational Science. 10.1016/j.procs.2019.08.193
2
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
177
There are two main approaches for events prediction in social networks. The first one is centered around the user activity patterns. User posts are considered as active responses to changes in the environment, and users themselves act as sensors [11]. Users friends list and previous check-ins are used to identify the next locations with the highest probability. The standard approach is to consider the user-location interaction as a link [12] since it allows to use various techniques starting from network analysis to machine learning for link prediction. Nevertheless, this approach limits forecasting systems to a set of particular users only and prediction of the state for some area becomes a hard task. Another way to forecast events or social network state is to predict changes in the popularity of a specific topic. These methods rely on text analysis and feature extraction. Although such techniques allow predicting an emergence of some trend in advance accurately, they have several serious drawbacks. First, some trended topics may not be related to an actual event; for example, it could be an active discussion or a flash mob. That is why all detected trends should be additionally checked, and list of observing topics should be specified. Secondly, such narrowing of monitoring domain could result in the absence of some events in the list of topics. This could happen in case when events of a particular type are unexpected in the studied area, i.e. street concerts, protests, etc. Thereby, there are two different issues, which should be taken into account for event prediction using social networks data. To avoid disadvantages of user- and content-based approaches, we propose a method for the city state prediction, which can determine the next state independently from a predefined message context or list of monitored users. Our method relies on a historical information of spatial activity in a social network. We take into consideration only spatiotemporal distribution of posts; usage of only geographical and temporal data allows to forecast changes in users behavior on a city scale. The method is based on creating dense grids for every hour of the available data, filling cells of these grids with aggregated statistics about users activity, and using grids for several previous hours to predict the next state. As a result, the model predicts the activity level in each cell for the next hour. Representation of a region state as a spatial grid opens doors to modern approaches for prediction such as machine learning and deep learning techniques [2]. In this work, we use Convolutional Neural Network model where city state is interpreted as a two-dimensional matrix. Convolutional neural networks (CNN) is well-known approach proven to be efficient for image classification and pattern recognition; they are also actively used for event detection and next frame prediction for video data. In this approach image is represented as a two-dimensional matrix where each cell corresponds to a pixel of the image and video is a sequence of such matrix. Since in our case, we also represent data as a matrix, and our primary goal is to predict the next state based on spatiotemporal patterns of historical data that is why we used CNN model for city state prediction. For a thorough evaluation of the developed approaches, we collected a large dataset of Instagram posts in the New York City area from 2010 to 2018. The resulting dataset contains more than 62 million posts in 114,000 locations. This allowed us to create baseline adaptive geogrids for the full year and investigate the efficiency of our solution in several experiments devoted to determining a precision rate and a recall rate. Experimental results show that urban state forecasting model achieves the recall of 69.4%.
2. Related works The modern studies dedicated to social network analysis moved one step further from the event detection to the event prediction [6]. In [1], the authors pointed out that in the modern era it is crucial to be proactive and be prepared for events which do not happen yet. This is extremely important for all sort of emergency events: natural disasters or criminal activity. However, information about cultural and fun events is also relevant for the municipal services and should be predicted as well. The most common way to predict events is to associate the event with a specified list of keywords or topics; these methods are focused on forecasting upcoming trends. For instance, in [18] Zhao et al. used the list of 124 predefined keywords related to flu for event prediction using Twitter. Usage of such wide list allowed to predict flu outbreaks with 94% of precision and the recall was 84% on the state level where precision is the percentage of the predictions which are actual events and recall is fraction of actual events which was predicted by model. Despite the fact that proposed approach achieves better results on the higher scale such as region and country level, other experiments showed that on the city level the recall value exceeds 50% only for three countries out of eight. Their own method based on Hidden
178
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
3
Markov Model and presented in [17] demonstrates better results with 69% of recall for flu outbreaks detection in New York and 70% of recall for civil unrest events in Mexico. In [13], the list of 72 terms was used to forecast flu outbreaks by Twitter stream. Data from Centers for Disease Control and Prevention was used to evaluate the results. Similar to Zhaos work authors used one week time interval for the prediction of flu since this time frame used in control dataset. The event considered as predicted if there was an alert within seven days prior to the event. The true positive rate for developed method is 0.55. Thus, the one of major issues in the event prediction is to predict all occurring events which is hard for term-based methods since they require excessive list of keywords for each of possible event. In [5] Huang et al. proposed a novel method to ensure three essential issues in event prediction: (a) the earliest possible prediction of the event, (b) observing dynamic of the event and (c) extraction of coherent words to the event. In presented approach authors interpreted the event as a topic which is actively discussed in some microblog. They developed emerging topic tracking algorithm (ETT) for monitoring of emerging, growing and fading states of the topic. This algorithm utilizes spatial and temporal features to monitor the topic emergence and its following life-cycle. Authors used Sina Weibo microblog service to demonstrate the effectiveness of their methods. Despite experiments showed 77% accuracy on the topic emergence, this method detects the event only with its beginning. Thus, the prediction task was remained unsolved by authors. The ensemble of models for crime events prediction using Foursquare data was presented in [10]. Rumi et al. split the day into three-hours periods and tried to predict the occurrence of crime event in the next time interval. The historical, geographic, demographic and dynamic features were used in the model and results showed that the use of dynamic features leads to improvement of the accuracy of predictions. Authors chose New York City and Brisbane as their target cities for experiments and achieved up to 99% accuracy for Brisbane and 77% for New York. These results can be explained by the fact that Brisbane is a safer city than New York. During the experiments, authors compared ensemble with the performance of each model separately. It was shown that Random Forest and Neural Network perform with the comparable results to the ensemble of models. However, the presented method relies not only on Foursquare data but also on historical data about crimes and demographic data which are not available for every region. In addition to that, this approach is focused on type of events crime events and does not cover other possible events. Thus, prediction of city state requires novel methods which are capable of forecasting activity in geographic area in general. The applicability of Deep Neural Network (DNN) to event prediction was also demonstrated in [15]. Authors used linear Support Vector Machine, Decision Tree, Random Forest and DNN for vehicle crash prediction based on demographic and weather data. It was shown that DNN achieves the highest accuracy and usage of the spatial features such as road network connectivity improves this result. In [4] it was shown that event detection methods can be successfully used for prediction because the modern methods of event retrieval identify event up to two days before the news agencies, the earthquake in Latin America was taken as example and even the worst performing method detected event 2.04 days ahead of news reports. The CNN architecture was developed by Farajidavar et al. [3] for early event detection in Twitter. The main goal of this research was to identify type of occurring event correctly. To achieve that authors used dataset labeled in 7 major event categories: food, environment, social, sport, crime, cultural, and transport events. The model performed with 81% accuracy on the event extraction. However, this method is based on text features, so it cannot be extrapolated on the geographic areas with any preferable language but English. Moreover, presented model is limited by aforementioned event classes so it might fail in case of mixed-classes events or events which are hard to range in the certain class, for example, local fairs. Existing methods for event forecasting for the most part focus on predicting popularity of topic represented by a list of keywords. To achieve the satisfactory results, such methods require long lists of terms (in some cases more than 100) associated with topic. If these terms are commonly used words, it becomes hard to identify a true location of the event. Thus, novel methods utilizing temporal and spatial features of data becoming more relevant. Approaches based on deep learning even though they were designed for specific types of events show promising results and in this work, we use convolutional neural network to predict events of diverse types for following hour.
4
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
179
Fig. 1. Data processing scheme
3. Dataset For the event prediction, we used the data from Instagram. We selected a time period from 1 January 2017 till 18 April 2018 and gathered data from 108846 locations. We used the first 12 months for model training, and all the data for 2018 was used for validation and model evaluation. Since we train our model on the full year, we can be sure that seasonal trends are taken into consideration. We select five-hours timeframe as input data for prediction, this size of time widow was chosen in order to balance between daily dynamics and current trend. Analysis of the impact of different time windows on the model performance is discussed in next section. In addition to that, during aforementioned period the number of Instagram profiles have grown approximately on 300 millions of users, and by the end of June, the number of active users has reached more than 1 billion profiles. So, by achieving satisfying results with our model, we can expect sufficient results from our model with following Instagram expansion. The basic scheme of data processing is presented in Figure 1. Due to the computational limitations, we cut the region of interest for prediction to the city center, so the final rectangle was bounded by coordinates [-74.058257, 73.856257, 40.63484, 40.82884], the covered area is shown on Figure 1, left. We used only timestamp and coordinates of each post. On the first step, we constructed a dense grid with spacing equal to 0.0005 and placed all posts for each hour separately. Thus, we obtained the initial grid with activity statistics for each hour of each day for the whole time period where cell value corresponds to a number of photos posted during that specific hour. In our approach, we use five previous hours to predict the following one (Figure 1, middle). So, we construct three-dimensional tensor with shape [5x404x388] where the first dimension represents previous time slices and second and third dimensions correspond to latitude and longitude, respectively. These tensors are used as an input data for our prediction model and similar one-dimensional tensor for the following hour (Figure 1, right) represents a desirable result of prediction. Thus, our model is capable of forecasting future users activity starting from the sixth hour in the obtained data. 4. Method description As it was previously explained, Convolutional Neural Networks is a promising and natural approach for this kind of a task. In our previous study, we have shown that CNN model can predict an urban area state with up to 99% accuracy on the average [8]. Thus, in this work, we decided to use the network architecture, which performed best. The deep network architecture is presented in Figure 2; this architecture is based on six three-dimensional convolutional layers with ELU as an activation function. As can be seen from the schema, input data convolves by all three dimensions: width, height, and depth that represents time. To obtain the final grid coverage the same area as in the input data, we switch the kernel size to k = (1,3,3) when time dimension achieved size 1 and keep this value till the last layer. The stride and padding value were similar for each layer and equal s = 1 and p = (0,1,1) respectively. The L1 loss was used in the model with ADADELTA optimizer algorithm [16] with starting learning rate lr = 0.5. The choice of these
180
Ksenia D. et al.Computer / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et Mukhina al. / Procedia Science 00 (2019) 000–000
5
Fig. 2. Proposed CNN architecture
parameters was determined by evaluation of model performance, and it was explained in details in our previous work [8]. For the performance evaluation, we used the same function, which is an average deviation percentage from the ground truth dense grid. Since dense grid represents the actual activity level in the city, it has some low and extremely high activity areas with sharp transitions. For accurate event detection, it is important to correctly predict areas with high and medium activity areas since they may be event indicators. Nevertheless, it is also important to predict areas with zero activity to reduce the number of assumed event candidates that should be checked. This following function was designed to balance between accurate prediction of high and low activity areas:
ev =
N 1 | yi j − yˆ i j | · 100%, N i, j yˆ i j
(1)
where yi j is a prediction matrix element, and yˆ i j is the matching cell of ground truth dense grid. In contrast to our previous work, we focused on the city center and used the dense grid with half as large step size. First, since city center usually attracts more people and more events take place in center comparing with districts, with smaller step size we can achieve better accuracy since we will get smooth transitions between high and low activity areas. Second, step size equal to 0.0005 matches the size of the initial statistical grid for event detection method. Thus, we will be able to compare the result obtained by event detection with our predictions. Since we changed the grid and step size, the network was trained for this area separately. In addition to that, we conducted a series of experiments to identify an optimal time window length; for each launch, network was trained for 100 epochs. We used the workstation equipped by NVIDIA Tesla P100, and the model was implemented with the help of PyTorch framework [9]. The results of experiments for parameter adjustment are demonstrated on Figure 3. Time window size was varied between 3, 5, and 7 hours in order to achieve the balance between the current situation in the city and repeating daily dynamics. As can be seen from the plot the average deviation shows almost no improvements during the training for seven hours with rising in error at the end, meantime, for three hours there is a constant upward trend. In the same time, the average deviation for the five-hour window demonstrated strong declining trend and reached its minimum at 87th epoch. Thus, since model performs best with a five-hour time window, we selected this time frame for the following analysis. Finally, the described combination of model parameters with this size of time window allows achieving the best average deviation which is 0.95% for the center of New York City. The result of prediction model represents a dense tensor of activity in the area, where the value in each cell equals to the expected number of posts in certain geographic area. Since the prediction model relies on spatial and temporal data only, it is impossible to forecast the content or type of events. But based on previous activity dynamics, neural network is able to highlight areas with a high probability of some event in the future. Result matrix can play the role of
6
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
181
Fig. 3. Effect of different time-window on CNN performance
statistical grid and be used as an input data for anomalies search. However, resulting tensor consists of float number, meanwhile number of posts is an integer number. So, first, the output data is rounded up to the closest integer. On the last step, predicted dense grid with activity values is used in the event detection pipeline. Thus, we obtain the list of candidates as a result of event prediction algorithm. In next section we compare results obtained by our prediction model with actual events identified by our event detection method. 5. Experiments In this section we analyze the quality of results of event prediction model. For the evaluation of the prediction model, we decided to pick the same two weeks from the previous study dedicated to event detection [14]: active week (12-18 March 2018) and non-active week (19-25 February 2018). This event detection method allows to achieve up to 100% of entertainment and beauty events with 77.1% of precision on average on various categories. Since there is no reliable source of all events in the city (news media usually covers only urgent events; meanwhile concert halls put a schedule on their websites), results of event detection method become the complete list of past events. To compare the results obtained by adaptive geogrids, the historical grids were constructed for the city center based on the same dense grid, which is used in the CNN. We ran anomalies search method on the prediction model results and obtained the list of candidates for these two weeks. In Figure 4, the heatmap for predicted activity matrix is presented. As can be seen from the map even in the city center high and low activity areas differ drastically. Even though in some areas predicted activity level does not precisely match with an actual number of posts, in the majority of cases CNN was able to correctly predict zero activity areas as well as the most active places. Moreover, the average deviation is equaled to 0.95%, i.e. CNN based on three-dimensional convolutional layers ensures 99.05% accuracy for the center of the New York City. Since in this experiment we predict only event candidates it is more important to get an accurate prediction of all detected events than to decrease the false positive rate and predict event only. In Table 1, results of the model are presented, we compared predicted candidates with actual events obtained by adaptive geogrids during experiments in previous section. Table 1. Summary of event prediction results
Active week Non-active week Total
True events
Detected events
%
87 96 183
58 69 127
66.7% 71.8% 69.4%
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
182
7
Fig. 4. Comparison of the predicted level of activity (left) with ground truth data (right)
The event is considered as correctly predicted if there is a respective candidate obtained from the prediction results by adaptive geogrids. Since with different amount of data we get different structure of ConvTree we looked for overlapping between leaves where an actual event was detected and where a candidate was obtained. Recall value is 69.4%, which complies with the best result for recall in this area [17, 18]. In the prediction model we do not have context information about predicted posts, and consequently, we cannot distinct high active area from actual events like we do in the event detection method. Even though the number of candidates highly exceeds the number of actual events (Table 2), prediction model reduces the number of leaves for further analysis as well as geographic area of possible events by 33%. Thus, our model can be successfully used in decision support systems or early-warning systems where it is important to have knowledge of upcoming events in prior. Table 2. Event candidates summary
Event detection Event prediction
Actual events
False candidates
%
183 127
2100 1402
8.7% 9%
It is important to note here that we consider event as predicted if we have a candidate for the exact same hour as it is in event detection algorithms result. Thus, it is possible that some event was correctly predicted not in the beginning of the event but during following hours. For example, concerts last for several hours and exhibitions take places for several months. To overcome this issue, we need to obtain the full list of events with their durations. One of possible way to create such dataset is to use our event detection method and merge the same event from different hours. It can be done by analyzing co-occurrence of the same hashtags in geographically close leaves from different grids. On the other side, in this work we try to forecast events for following hour. In the meantime, some events such as holidays parades or protests are planned in advance, so they can be predicted by analyzing year patterns or context of messages. It was shown that in social media first language is prevalent [7], thus, one of the important challenge will be to develop methods for different languages. However, the main goal of this work was to predict as many events of various nature as possible so in this method we did not use additional data on purpose.
8
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
183
6. Conclusion In this work we presented a prediction model based on a convolutional neural network. This model allows to predict users activity in the area with 1% average deviation from the ground truth data and achieve 69.4% recall on the event prediction task. Nevertheless, we see two directions to enhance the proposed forecasting method. First, we need to decrease false positive rate and to lower the number of detected candidates. It can be achieved by using attention techniques in the neural network. This improvement would also impact on the performance in term of computation time and memory consuming. Second direction is to try more complicated models for the prediction. In this work, we used only convolutional layers, but combination of recurrent neural network and convolutional neural network approaches may identify more complicated spatiotemporal activity patterns. In addition to that, we are going to use context from data for the prediction because for some cultural event people are very excited and start to actively post several hours prior to event. Nonetheless, event forecasting methods based on the spatiotemporal features of data showed sufficient results for their tasks. The experiments demonstrated that usage of three-dimensional convolutions for event forecasting performs on the same level as the best and most complicated methods in this area. Acknowledgements This research is financially supported by the Russian Science Foundation, Agreement #18-71-00149. References [1] Dencik, L., Hintz, A., Carey, Z., 2018. Prediction, pre-emption and limits to dissent: Social media and big data uses for policing protests in the United Kingdom. New Media and Society 20, 1433–1450. doi:10.1177/1461444817697722. [2] Do, T.H., Nguyen, D.M., Tsiligianni, E., Cornelis, B., Deligiannis, N., 2018. Twitter User Geolocation Using Deep Multiview Learning, in: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 6304– 6308. URL: http://homepages.vub.ac.be/{~}ndeligia/pubs/icassp{_}18{_}tien.pdf, doi:10.1109/ICASSP.2018.8462191, arXiv:1805.04612. [3] Farajidavar, N., Kolozali, S., Barnaghi, P., 2017. A Deep Multi-View Learning Framework for City Event Extraction from Twitter Data Streams URL: http://arxiv.org/abs/1705.09975, arXiv:1705.09975. [4] Hua, T., Chen, F., Zhao, L., Lu, C.T., Ramakrishnan, N., 2016. Automatic targeted-domain spatiotemporal event detection in twitter. volume 20. GeoInformatica. URL: http://dx.doi.org/10.1007/s10707-016-0263-0, doi:10.1007/s10707-016-0263-0. [5] Huang, J., Peng, M., Wang, H., Cao, J., Gao, W., Zhang, X., 2017. A probabilistic method for emerging topic tracking in Microblog stream. World Wide Web 20, 325–350. URL: https://doi.org/10.1007/s11280-016-0390-4, doi:10.1007/s11280-016-0390-4. [6] Kursuncu, U., Gaur, M., Lokala, U., Thirunarayan, K., Sheth, A., Arpinar, I.B., 2018. Predictive Analysis on Twitter: Techniques and Applications , 1–37URL: http://arxiv.org/abs/1806.02377, doi:10.1007/978-3-319-94105-9_4, arXiv:1806.02377. [7] Magdy, A., Mokbel, M.F., Elnikety, S., Nath, S., He, Y., 2014. Mercury: A Memory-Constrained Spatio-temporal Real-time Search on Microblogs, in: Proceedings - International Conference on Data Engineering. URL: https://www.microsoft.com/en-us/research/ wp-content/uploads/2016/02/samehe-icde2014.mercury.pdf. [8] Mukhina, K.D., Visheratin, A.A., Mbogo, G.K., Nasonov, D., 2018. Forecasting of the Urban Area State Using Convolutional Neural Networks. 2018 23rd Conference of Open Innovations Association (FRUCT) , 268–275URL: https://www.theverge.com/2018/6/20/17484420/. [9] Paszke, A., Chanan, G., Lin, Z., Gross, S., Yang, E., Antiga, L., Devito, Z., 2017. Automatic differentiation in PyTorch. Advances in Neural Information Processing Systems 30 . [10] Rumi, S.K., Deng, K., Salim, F.D., 2018. Crime event prediction with dynamic features. EPJ Data Science 7, 43. URL: http://dx.doi. org/10.1140/epjds/s13688-018-0171-7, doi:10.1140/epjds/s13688-018-0171-7. [11] Sakaki, T., Okazaki, M., Matsuo, Y., 2010. Earthquake Shakes Twitter Users: Real-time Event Detection by Social Sensors. Proceedings of the 19th International Conference on World Wide Web doi:10.1145/1772690.1772777, arXiv:0808.0743v3. [12] Scellato, S., Mascolo, C., 2011. Exploiting Place Features in Link Prediction on Location-based Social Networks Categories and Subject Descriptors, in: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1046–1054. URL: http://www1.se.cuhk.edu.hk/{~}seem5010/slides/kdd2011{_}placefriends.pdf, doi:10.1145/2020408.2020575. [13] Shao, M., Li, J., Chen, F., Huang, H., Zhang, S., Chen, X., 2017. An Efficient Approach to Event Detection and Forecasting in Dynamic Multivariate Social Media Networks, in: Proceedings of the 26th International Conference on World Wide Web - WWW ’17. doi:10.1145/ 3038912.3052588. [14] Visheratin, A.A., Mukhina, K.D., Visheratina, A.K., Nasonov, D., Boukhanovsky, A.V., 2018. Multiscale Event Detection Using Convolutional Quadtrees and Adaptive Geogrids, in: Proceedings of the 2Nd ACM SIGSPATIAL Workshop on Analytics for Local Events and News, ACM, New York, NY, USA. pp. 1:1—-1:10. URL: http://doi.acm.org/10.1145/3282866.3282867, doi:10.1145/3282866.3282867.
184
Ksenia D. Mukhina et al. / Procedia Computer Science 156 (2019) 176–184 K. Mukhina et al. / Procedia Computer Science 00 (2019) 000–000
9
[15] Yuan, Z., Tamerius, J., 2017. Predicting Traffic Accidents Through Heterogeneous Urban Data: A Case Study. The 6th International Workshop on Urban Computing URL: https://doi.org/10.475/123{_}4. [16] Zeiler, M.D., 2012. ADADELTA: An Adaptive Learning Rate Method. arXiv URL: http://arxiv.org/abs/1212.5701, doi:http: //doi.acm.org.ezproxy.lib.ucf.edu/10.1145/1830483.1830503, arXiv:1212.5701. [17] Zhao, L., Chen, F., Lu, C.T., Ramakrishnan, N., 2015. Spatiotemporal Event Forecasting in Social Media. Proceedings of the 2015 SIAM International Conference on Data Mining , 963–971URL: http://epubs.siam.org/doi/10.1137/1.9781611974010.108, doi:10.1137/ 1.9781611974010.108. [18] Zhao, L., Chen, F., Lu, C.T., Ramakrishnan, N., 2017. Multi-resolution spatial event forecasting in social media. Proceedings IEEE International Conference on Data Mining, ICDM , 689–698URL: http://www.nvc.cs.vt.edu/{~}ctlu/Publication/2016/ ICDM-Proceedings-2016.pdf, doi:10.1109/ICDM.2016.75.