- Email: [email protected]
Crowd Data Visualization and Simulation
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2018) 000–000 Procedia Computer Science (2018) 000–000 Procedia Computer Science 13900 (2018) 622–629
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
The International Academy of Information Technology and Quantitative Management, the Peter The International Academy Kiewit of Information andNebraska Quantitative Management, the Peter Institute,Technology University of Kiewit Institute, University of Nebraska
Crowd Data Visualization and Simulation Crowd Data Visualization and Simulation
Leonel Antonio Toledo Diazaa, Ivan Rivalcoba Rivasbb, Krely Rodriguezcc, Isaac Rudominaa Leonel Antonio Toledo Diaz , Ivan Rivalcoba Rivas , Krely Rodriguez , Isaac Rudomin a Barcelona Supercomputing center , Career Jordi Girona 29, Barcelona 08034, Spain Girona CDMX, 29, Barcelona 08034, Spain Instituto Tecnol´ocenter gico de, Career GustavoJordi A. Madero, M´exico. b Instituto Tecnol´ c Tecnologico ogico de Gustavo A. Madero, CDMX, exico. Nacional de Mexico / Instituto Tecnologico deM´ Minatitlan c Tecnologico Nacional de Mexico / Instituto Tecnologico de Minatitlan
a Barcelonab Supercomputing
Abstract Abstract Simulating large and complex urban environments for real-time simulation poses multiple challenges which are not trivial to Simulating large andgathering complexinformation urban environments for for real-time poses management multiple challenges are not trivialand to solve, for instance, of geometry cities simulation to be displayed, of largewhich quantities of data solve, for instance,the gathering information of the geometry for cities to be displayed, management large of data fetching efficiently information needed for simulation and visualization of virtual pedestriansofare justquantities some examples of and the fetching efficiently challenges we face. the information needed for the simulation and visualization of virtual pedestrians are just some examples of the challenges we face. In this work we present the results we have achieved so far using a system that addresses problems such as data visualization In this we present the results we have achieved so far using a system thatintegration addresses of problems visualization using levelwork of detail techniques, GPU processing for efficient computation and the real datasuch intoas thedata simulation such using level of detail techniques, GPU processing for efficient computation and the integration of real data into the simulation such as GPS data and real 3D cities that are generated using tools such as WRLD-3D plug-in. This method has been proven powerful as GPS to data and large real 3D citiescomposed that are generated usingoftools such as WRLD-3D plug-in. This method hasrates been(atproven powerful enough handle crowds by thousands agents in urban environments at interactive frame least 30 frames enough to handle large composed thousands of agents in urban environments at interactive rates (at frames per second or 33.33 mscrowds per frame) withoutbycompromising the visual quality and the complexity of theframe behaviors by least using30publicly per second or 33.33 ms per frame) without compromising the visual quality and the complexity of the behaviors by using publicly real world data such as city topography and pedestrian GPS data. real world data such as city topography and pedestrian GPS data. c 2018 Authors. Published Published by by Elsevier Elsevier B.V. © 2018 The The Authors. B.V. c 2018 The Authors. Published by Elsevier B.V. This is an open access article under license This is an open access article under the the CC CC BY-NC-ND BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) (http://creativecommons.org/licenses/by-nc-nd/4.0/) This review isreview an open access article under thethe CCscientific BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer under responsibility of of the scientific committee of The International Academy of Information Technology and QuanPeer under responsibility committee of The International Academy of Information Technology and Peer review under responsibility of the scientific committee The International Academy of Information Technology and Quantitative Management, the Peter Institute, University ofofNebraska. Quantitative Management, the Kiewit Peter Kiewit Institute, University of Nebraska. titative Management, the Peter Kiewit Institute, University of Nebraska. Keywords: Crowd simulation; finite state machine (FSM); Behaviors; Level of Detail; Keywords: Crowd simulation; finite state machine (FSM); Behaviors; Level of Detail;
1. Introduction 1. Introduction Simulating the motion of realistic and dense crowds is still an interesting challenge for the computer graphics the motion of dense crowds still an interesting for the the computer graphics andSimulating animation community, therealistic standardand approach to solveisthis problem usually challenge involves dividing simulation into and animation community, the standard approach to solve this problem usually involves dividing the simulation into microscopic and macroscopic scales of the agents involved and treat the elements using techniques that are commonly microscopic and macroscopic scales of the agents involved and treat the elements using techniques that are commonly associated with particle simulation and are given simple behavior rules to mimic reality as much as possible. associated with particle simulation and are given simple behavior rules to mimic reality as much as possible. ∗
∗
Leonel Toledo. Leonel address: Toledo. [email protected] E-mail E-mail address: [email protected]
c 2018 The Authors. Published by Elsevier B.V. 1877-0509 1877-0509 © 2018 Thearticle Authors. Published by B.V. (http://creativecommons.org/licenses/by-nc-nd/4.0/) c 2018 1877-0509 The Authors. Published by Elsevier Elsevier license B.V. This is an open access under the CC BY-NC-ND This isis an an openaccess accessarticle articleunder under the BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This CCCC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer reviewopen under responsibility of the scientific committee of The International Academy of Information Technology and Quantitative ManagePeer review under responsibility of the scientific committee of The International Academy of Information Technology and Quantitative Peer review under responsibility of the scientific committee of The International Academy of Information Technology and Quantitative Management, the Peter Kiewit Institute, University of Nebraska. Management, the Peter Kiewit Institute, University of Nebraska. ment, the Peter Kiewit Institute, University of Nebraska. 10.1016/j.procs.2018.10.201
2
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629
623
Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 1. Urban environment created using synthetic data.
Even tough hardware capabilities are surpassed constantly by each new generation of hardware, the number of polygons, memory and computational resources are not enough to properly simulate dense crowds composed by millions of characters. This cannot be achieved without the aid of rendering techniques such as Level of Detail (LOD). Interactive virtual crowds require high-performance simulation, animation and rendering techniques to handle numerous characters in real-time. These characters must be believable in their actions and behaviors. Real-time crowd simulation brings different challenges compared with systems that involve a small number of interacting characters (i.e. most contemporary computer games) and non-real-time (i.e. crowds seen in movies or visualizations of crowd evacuations after off line model computations). In comparison with single agent simulations, the main conceptual difference is the need for efficient variety management at every level, whether is visualization, motion-control, animation or sound rendering [1]. Many of the techniques that are involved in area are faced with memory and data acquisition constraints, in this work our main contribution is the integration of state of the art techniques that are used to visualize large volumes of data in real time as well as the management of data that is publicly available for us to construct real cities with the information of the pedestrians and the routes they follow or how they behave under certain circumstances. The rest of this work is organized as follows: a brief overview of the most relevant work that was taken into consideration for building our system is presented in section 2. Section 3 describes in detail how we construct, compute and simulate the most relevant data within the urban environment and the different techniques that are employed to efficiently display the agents at interactive frame-rates. The results of the experiments performed are presented in section 4 and finally we conclude by presenting the limitations of the current state of the system and future work in section 5.
2. Related Work Large pedestrian agglomeration is a common phenomenon that is observable in many cities and public places around the world. There has been some extensive research in this area, and many representations and approaches are considered, some are focused on rendering while other authors show more interest toward the data acquirement and simulation needed to reproduce this phenomena. In this section we discuss some of the most important work that was taken into consideration when implementing the system.
624
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
3
2.1. Level of Detail Simulating dense crowds that are composed of hundreds of thousands virtual humans is impossible without the aid of Level of Detail (LOD) techniques. The requirement in interactive systems for real-time frame rates means that a limited number of polygons can be displayed by the graphics engine in each frame of a simulation. Therefore, meshes with a high polygon count often have to be simplified in order to achieve acceptable display rates. This can be achieved by representing the object at several levels of detail with fixed polygon count or simplified based on the current view, thus allowing a smooth transition from lower levels to higher levels of detail [2]. The work of Toledo et al. [3] shows a hybrid hierarchical level of detail technique based on two complementary data structures that handle level of detail using the distance to the viewer as the main criteria for the discretization of the elements in the crowd. 2.2. Geographic Information Systems Visualization applications such as 3D globe based interfaces, navigation systems presenting a 3D perspective are increasing rapidly due to the recent developments in geographic information systems and data acquisition. This has created the need to develop algorithms to reconstruct 3D data using 2D objects [4]. It is important to address that modeling and managing 3D data in GIS is a non-trivial task. The work of Essen [5] describes a method used to produce 3D maps taking as a base a 2D city maps which contains relevant features, for instance this maps can be created by collecting height data of each of the buildings and using textures to represent facades. We extend this work by implementing a pipeline that allows us to extract urban and city information to create complex environments using real data and combining it with an interactive crowd. The work of Slingsby et al. [6] describes the necessity of 3D city models as important resources for planning and decision-making and the importance that has including navigable space for pedestrians, in other words, space within buildings and the connection between interior and exterior space. In this work, they are incorporating a semantic system, whereas, our proposed pipeline constructs the environments using information that is collected directly from open street maps and incorporated within our simulation. The work of Thomsen et al. [7] introduces a general approach for modeling topology in 3D GIS. and addresses the problem of using real 3D data in comparison with traditional 2D or 2.5D and how the context of topological abstraction influences the final result, depending on the operations applied to a certain set of data. Using a cell layout hierarchies are created and geometry can have a mesh representation. 2.3. Pedestrian Simulation Realistic crowd simulation involves a lot of components and variables such as group behavior, some kind of cognitive modeling, motion planning, rendering, trajectory computations and many others, the work by Guy [8] proposes an algorithm for simulating large-scale crowds which is based on the principle of least effort by using a mathematical model for representing the amount of effort that is expended by each agent. The work of Pelechano et al. [9] shows an approach on simulating the motion of realistic, large, dense crowds of autonomous characters moving in a natural manner and dynamically changing virtual environments. The work of [10] explains how Big Data management and analysis is vital in multiple scientific disciplines and how several areas of knowledge handle the different challenges that arise given a certain context. 3. Crowd Data and Simulation Simulating and populating large virtual environment is not a simple task, there are several problems that must be addressed and solved to properly display the crowd and achieve results that closely resemble reality. We present a system capable of handling several thousands of varied animated characters within a crowd. These characters are designed to have geometric, color animation and behavior variety, nevertheless when a crowd becomes bigger, more memory is needed and is often difficult to achieve this objective. We have designed this system taking into consideration several constraints, populating and simulating different scenarios in real-time using data that matches
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629
4
625
Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 2. A simplified XML that controls the distribution of elements inside the crowd.
reality, for example GPS data for the pedestrians to follow. The implementation of the system is further explained in the work of Toledo et al [11].
3.1. Populating the environment Constructing virtual environments is a task that can take a lot of time, effort and resources even for well established groups of animators and designers. Hand crafting every single member of the crowd or asset that is involved within the simulation is virtually impossible. We have developed an XML-based architecture that we use to create the general distribution of agents in the environment. Figure 3.1 shows a simplified version of what we use to parse the most important features of the crowd. By using this file we are able to describe the crowd using as many parameters as we want, for example we can determine how many elements will take part of the simulation and how many groups are involved, in this case we have a crowd composed by 5000 elements which are divided into different categories. In this particular case we determined to have a 40 per cent of male population of the global size of the crowd and that percentage is defined to have visual variety. We use this file to configure parameters not only for visualization but for simulation as well. For example we can go deeper and determine that a varied group of females can have a predominant age group or use a particular set of winter clothes and load the proper assets. But as mentioned previously, the file is not limited to visualization; we can extend the functionality of the system and determine how each agent will behave within the simulation, we can determine that a certain category simulation will be updated using social forces while other group might use GPS data, fuzzy logic or finite state machines just to state some cases. The system is robust enough to handle this kind of configurations as long as the proper shader is provided. Figure 3.1 shows the results of creating a urban environment by using the previously discussed techniques, by using the WRLD plug-in we collect the geographic information required, for our approach we consider two different techniques that we combine; first we collect data from GPS traces that describe routes that pedestrians take within the city this includes data such as latitude, longitude, elevation and a time stamp from when the sample was collected. From this moment on our agents can follow each of the routes and be animated accordingly, nevertheless it is important to address that memory access and storage can be an important bottleneck in the simulations, since ideally every agent should have one copy of the route that must follow, which makes the memory requirements grow significantly. To
626
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
5
Fig. 3. Results of creating the virtual environment.
Fig. 4. Instances of the elements of the crowd are created using dynamic materials.
avoid this we create GPS traces that work as guides to the agents and we create different instances at different points in time to mitigate the issue. Figure 3.1 shows how we integrate the previously discussed techniques and how the variety is achieved, compared to the work of Millan et al.[12] and Kavan et al. [13] we achieve the same frame rates without the memory constraints, for instance those approaches needed, roughly 12 megabytes for each character representation, animations and views, that means that a crowd composed by 1000 visually unique characters would need 12,000 megabytes of memory. We can achieve comparable results in terms of frame-rates without memory constraints, crowds of 10,000 agents can run using less than 300 megabytes of memory.
6
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629
627
Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 5. A detailed description of all the data that is needed to render a single frame of the simulation.
3.2. Behavior and motion planning for pedestrians As mentioned previously, the way the agents move and react within the simulation is determined by mixing datadriven steering and group social forces. We use a database of preprocessed trajectories of pedestrians which we store as a set of vectors that encode the steering motion of pedestrians interacting with each other in public spaces. This data is used to generate steering actions given certain inputs, which can be a set of states that affect the surroundings of any virtual character. This set of actions is complemented using Helbing’s social group forces than might be found on real scenarios [14]. Nevertheless, pedestrian behavior is a very complex area of study that is not possible to model completely by using deterministic methods, at least to certain degree of realism. To solve this problem we incorporate fuzzy logic into the simulation, this way we are able to profile agents within the simulation, the system allows us to profile agents and assign values to a shared library of parameters that all characters inherit from. This way we can create two different profiles for all the elements in the simulation, the first is focused on the senses of each agent, such as vision range, maximum speed, weight, to state a few, and the second profile is oriented to the understanding of fuzzy concepts for each characters, for instance, agents may know what ”fast” or ”slow” are, but have completely different meaning regarding their physical capabilities. One of the main advantages of this method is that all agents have access to this knowledge and without major chances we can achieve a lot of variety in the crowd behavior.
4. Results For our experiments we executed the simulation and visualization process in a workstation with these characteristics: Intel Core i7-4810MQ CPU @ 2.80GHz 8, 16Gb of RAM, GeForce 880M with Gb of video memory. For a simulation in the city (Barcelona) with one thousand characters 4.3 million triangles are needed and 3.1 million vertices. We use 1115 draw calls and 39 batched draw calls. It takes a total of 300 megabytes of RAM memory, 112 megabytes of video memory and it takes 28 milliseconds to render each individual frame giving us a 35.71 frame rate. In average each frame has 18000 objects and the total scene is composed by 42767 objects. Figure 4 shows in detail every piece of data that is needed to render a single frame of the simulation, in this case the table is displaying the inputs needed to compute a mesh for any given zone in the urban environment, we do calculations using shaders in the GPU.
628
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
7
Fig. 6. Instances of the elements of the crowd are created using dynamic materials.
Figure 4, shows how the previously calculated mesh is transformed into an instance of the scene and elements are placed using both the geometric information of the system and the geographic coordinates that are associated with each of the elements in the simulation. 5. Conclusions and Future work As shown in this work, crowd simulation is a very complex area that involves many other disciplines, which have helped us to understand how to apply different techniques and handle date efficiently to simulate dense crowds in real-time Applications for virtual cities and complex environments range from research and educational purposes such as urban planning and simulation, to a wider area such as the movie and entertainment industry, security, traffic simulation, disaster planning and many more. With our approach, we have developed a robust system that can handle interactions within pedestrians and visualization of environments. As future work, it is important to extend the scope of the simulation and optimize the way we store and transfer our data to our agents, sometimes GPS traces can be heavy and slow to parse, and at this moment we are not able to process them in real time, meaning that we cannot simulate characters that use different routes that the ones we have already pre-computed. It would be interesting to launch the simulation using geo-spatial queries and adapt the level of detail algorithms we currently employ to further expand the simulation. Acknowledgements Special thanks to SECITI in Mexico for providing part of the founding for the present research, also we must express our gratitude for all the team that conforms the Barcelona Super Computing Center for facilitate the equipment and the infrastructure for the project. References [1] D. Thalmann, H. Grillon, J. Maim, B. Yersin, Challenges in Crowd Simulation, 2009 International Conference on CyberWorlds (2009) 1– 12doi:10.1109/CW.2009.23. URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5279720
8
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
629
[2] C. O’Sullivan, J. Cassell, H. Vilhjlmsson, J. Dingliana, S. Dobbyn, B. McNamee, C. Peters, T. Giang, Levels of detail for crowds and groups, Computer Graphics Forum 21 (4) (2002) 733–741. doi:10.1111/1467-8659.00631. URL http://dx.doi.org/10.1111/1467-8659.00631 [3] L. Toledo, O. De Gyves, I. Rivalcoba, I. Rudomn, Hierarchical Level of Detail for Varied Animated Crowds (2014). [4] P. van Oosterom, P. van Oosterom, S. Zlatanova, F. Penninga, E. Fendel, Advances in 3D Geoinformation Systems, 1st Edition, Springer Publishing Company, Incorporated, 2007. [5] R. van Essen, Maps Get Real: Digital Maps evolving from mathematical line graphs to virtual reality models, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008, pp. 3–18. [6] A. Slingsby, J. Raper, Navigable Space in 3D City Models for Pedestrians, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008, pp. 49–64. [7] A. Thomsen, M. Breunig, E. Butwilowski, B. Broscheit, Modelling and Managing Topology in 3D Geoinformation Systems, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008, pp. 229–246. [8] S. J. Guy, J. Chhugani, S. Curtis, P. Dubey, M. Lin, D. Manocha, Pledestrians: A least-effort approach to crowd simulation, in: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’10, Eurographics Association, Goslar Germany, Germany, 2010, pp. 119–128. URL http://dl.acm.org/citation.cfm?id=1921427.1921446 [9] N. Pelechano, J. M. Allbeck, N. I. Badler, Controlling individual agents in high-density crowd simulation, in: Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’07, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 2007, pp. 99–108. URL http://dl.acm.org/citation.cfm?id=1272690.1272705 [10] C. L. P. Chen, C.-Y. Zhang, Data-intensive applications, challenges, techniques and technologies: A survey on big data, Inf. Sci. 275 (2014) 314–347. [11] L. Toledo, I. Rivalcoba, I. Rudom´ın, Fuzzy and data-driven urban crowds, in: Computational Science - ICCS 2018 - 18th International Conference, Wuxi, China, June 11-13, 2018 Proceedings, Part III, 2018, pp. 280–290. doi:10.1007/978-3-319-93713-7\_23. URL https://doi.org/10.1007/978-3-319-93713-7_23 [12] E. Millan, I. Rudomin, Impostors and pseudo-instancing for gpu crowd rendering, in: Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, GRAPHITE ’06, ACM, New York, NY, USA, 2006, pp. 49–55. doi:10.1145/1174429.1174436. URL http://doi.acm.org/10.1145/1174429.1174436 ˇ ara, C. O’Sullivan, Polypostors: 2d polygonal impostors for 3d crowds, in: Proceedings of the 2008 [13] L. Kavan, S. Dobbyn, S. Collins, J. Z´ Symposium on Interactive 3D Graphics and Games, I3D ’08, ACM, New York, NY, USA, 2008, pp. 149–155. doi:10.1145/1342250. 1342273. URL http://doi.acm.org/10.1145/1342250.1342273 [14] M. Moussa¨ıd, N. Perozo, S. Garnier, D. Helbing, G. Theraulaz, The walking behaviour of pedestrian social groups and its impact on crowd dynamics., PloS one 5 (4) (2010) e10047. doi:10.1371/journal.pone.0010047. URL http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2850937
Available online at www.sciencedirect.com
ScienceDirect
Procedia Computer Science 00 (2018) 000–000 Procedia Computer Science (2018) 000–000 Procedia Computer Science 13900 (2018) 622–629
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
The International Academy of Information Technology and Quantitative Management, the Peter The International Academy Kiewit of Information andNebraska Quantitative Management, the Peter Institute,Technology University of Kiewit Institute, University of Nebraska
Crowd Data Visualization and Simulation Crowd Data Visualization and Simulation
Leonel Antonio Toledo Diazaa, Ivan Rivalcoba Rivasbb, Krely Rodriguezcc, Isaac Rudominaa Leonel Antonio Toledo Diaz , Ivan Rivalcoba Rivas , Krely Rodriguez , Isaac Rudomin a Barcelona Supercomputing center , Career Jordi Girona 29, Barcelona 08034, Spain Girona CDMX, 29, Barcelona 08034, Spain Instituto Tecnol´ocenter gico de, Career GustavoJordi A. Madero, M´exico. b Instituto Tecnol´ c Tecnologico ogico de Gustavo A. Madero, CDMX, exico. Nacional de Mexico / Instituto Tecnologico deM´ Minatitlan c Tecnologico Nacional de Mexico / Instituto Tecnologico de Minatitlan
a Barcelonab Supercomputing
Abstract Abstract Simulating large and complex urban environments for real-time simulation poses multiple challenges which are not trivial to Simulating large andgathering complexinformation urban environments for for real-time poses management multiple challenges are not trivialand to solve, for instance, of geometry cities simulation to be displayed, of largewhich quantities of data solve, for instance,the gathering information of the geometry for cities to be displayed, management large of data fetching efficiently information needed for simulation and visualization of virtual pedestriansofare justquantities some examples of and the fetching efficiently challenges we face. the information needed for the simulation and visualization of virtual pedestrians are just some examples of the challenges we face. In this work we present the results we have achieved so far using a system that addresses problems such as data visualization In this we present the results we have achieved so far using a system thatintegration addresses of problems visualization using levelwork of detail techniques, GPU processing for efficient computation and the real datasuch intoas thedata simulation such using level of detail techniques, GPU processing for efficient computation and the integration of real data into the simulation such as GPS data and real 3D cities that are generated using tools such as WRLD-3D plug-in. This method has been proven powerful as GPS to data and large real 3D citiescomposed that are generated usingoftools such as WRLD-3D plug-in. This method hasrates been(atproven powerful enough handle crowds by thousands agents in urban environments at interactive frame least 30 frames enough to handle large composed thousands of agents in urban environments at interactive rates (at frames per second or 33.33 mscrowds per frame) withoutbycompromising the visual quality and the complexity of theframe behaviors by least using30publicly per second or 33.33 ms per frame) without compromising the visual quality and the complexity of the behaviors by using publicly real world data such as city topography and pedestrian GPS data. real world data such as city topography and pedestrian GPS data. c 2018 Authors. Published Published by by Elsevier Elsevier B.V. © 2018 The The Authors. B.V. c 2018 The Authors. Published by Elsevier B.V. This is an open access article under license This is an open access article under the the CC CC BY-NC-ND BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) (http://creativecommons.org/licenses/by-nc-nd/4.0/) This review isreview an open access article under thethe CCscientific BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer under responsibility of of the scientific committee of The International Academy of Information Technology and QuanPeer under responsibility committee of The International Academy of Information Technology and Peer review under responsibility of the scientific committee The International Academy of Information Technology and Quantitative Management, the Peter Institute, University ofofNebraska. Quantitative Management, the Kiewit Peter Kiewit Institute, University of Nebraska. titative Management, the Peter Kiewit Institute, University of Nebraska. Keywords: Crowd simulation; finite state machine (FSM); Behaviors; Level of Detail; Keywords: Crowd simulation; finite state machine (FSM); Behaviors; Level of Detail;
1. Introduction 1. Introduction Simulating the motion of realistic and dense crowds is still an interesting challenge for the computer graphics the motion of dense crowds still an interesting for the the computer graphics andSimulating animation community, therealistic standardand approach to solveisthis problem usually challenge involves dividing simulation into and animation community, the standard approach to solve this problem usually involves dividing the simulation into microscopic and macroscopic scales of the agents involved and treat the elements using techniques that are commonly microscopic and macroscopic scales of the agents involved and treat the elements using techniques that are commonly associated with particle simulation and are given simple behavior rules to mimic reality as much as possible. associated with particle simulation and are given simple behavior rules to mimic reality as much as possible. ∗
∗
Leonel Toledo. Leonel address: Toledo. [email protected] E-mail E-mail address: [email protected]
c 2018 The Authors. Published by Elsevier B.V. 1877-0509 1877-0509 © 2018 Thearticle Authors. Published by B.V. (http://creativecommons.org/licenses/by-nc-nd/4.0/) c 2018 1877-0509 The Authors. Published by Elsevier Elsevier license B.V. This is an open access under the CC BY-NC-ND This isis an an openaccess accessarticle articleunder under the BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This CCCC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer reviewopen under responsibility of the scientific committee of The International Academy of Information Technology and Quantitative ManagePeer review under responsibility of the scientific committee of The International Academy of Information Technology and Quantitative Peer review under responsibility of the scientific committee of The International Academy of Information Technology and Quantitative Management, the Peter Kiewit Institute, University of Nebraska. Management, the Peter Kiewit Institute, University of Nebraska. ment, the Peter Kiewit Institute, University of Nebraska. 10.1016/j.procs.2018.10.201
2
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629
623
Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 1. Urban environment created using synthetic data.
Even tough hardware capabilities are surpassed constantly by each new generation of hardware, the number of polygons, memory and computational resources are not enough to properly simulate dense crowds composed by millions of characters. This cannot be achieved without the aid of rendering techniques such as Level of Detail (LOD). Interactive virtual crowds require high-performance simulation, animation and rendering techniques to handle numerous characters in real-time. These characters must be believable in their actions and behaviors. Real-time crowd simulation brings different challenges compared with systems that involve a small number of interacting characters (i.e. most contemporary computer games) and non-real-time (i.e. crowds seen in movies or visualizations of crowd evacuations after off line model computations). In comparison with single agent simulations, the main conceptual difference is the need for efficient variety management at every level, whether is visualization, motion-control, animation or sound rendering [1]. Many of the techniques that are involved in area are faced with memory and data acquisition constraints, in this work our main contribution is the integration of state of the art techniques that are used to visualize large volumes of data in real time as well as the management of data that is publicly available for us to construct real cities with the information of the pedestrians and the routes they follow or how they behave under certain circumstances. The rest of this work is organized as follows: a brief overview of the most relevant work that was taken into consideration for building our system is presented in section 2. Section 3 describes in detail how we construct, compute and simulate the most relevant data within the urban environment and the different techniques that are employed to efficiently display the agents at interactive frame-rates. The results of the experiments performed are presented in section 4 and finally we conclude by presenting the limitations of the current state of the system and future work in section 5.
2. Related Work Large pedestrian agglomeration is a common phenomenon that is observable in many cities and public places around the world. There has been some extensive research in this area, and many representations and approaches are considered, some are focused on rendering while other authors show more interest toward the data acquirement and simulation needed to reproduce this phenomena. In this section we discuss some of the most important work that was taken into consideration when implementing the system.
624
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
3
2.1. Level of Detail Simulating dense crowds that are composed of hundreds of thousands virtual humans is impossible without the aid of Level of Detail (LOD) techniques. The requirement in interactive systems for real-time frame rates means that a limited number of polygons can be displayed by the graphics engine in each frame of a simulation. Therefore, meshes with a high polygon count often have to be simplified in order to achieve acceptable display rates. This can be achieved by representing the object at several levels of detail with fixed polygon count or simplified based on the current view, thus allowing a smooth transition from lower levels to higher levels of detail [2]. The work of Toledo et al. [3] shows a hybrid hierarchical level of detail technique based on two complementary data structures that handle level of detail using the distance to the viewer as the main criteria for the discretization of the elements in the crowd. 2.2. Geographic Information Systems Visualization applications such as 3D globe based interfaces, navigation systems presenting a 3D perspective are increasing rapidly due to the recent developments in geographic information systems and data acquisition. This has created the need to develop algorithms to reconstruct 3D data using 2D objects [4]. It is important to address that modeling and managing 3D data in GIS is a non-trivial task. The work of Essen [5] describes a method used to produce 3D maps taking as a base a 2D city maps which contains relevant features, for instance this maps can be created by collecting height data of each of the buildings and using textures to represent facades. We extend this work by implementing a pipeline that allows us to extract urban and city information to create complex environments using real data and combining it with an interactive crowd. The work of Slingsby et al. [6] describes the necessity of 3D city models as important resources for planning and decision-making and the importance that has including navigable space for pedestrians, in other words, space within buildings and the connection between interior and exterior space. In this work, they are incorporating a semantic system, whereas, our proposed pipeline constructs the environments using information that is collected directly from open street maps and incorporated within our simulation. The work of Thomsen et al. [7] introduces a general approach for modeling topology in 3D GIS. and addresses the problem of using real 3D data in comparison with traditional 2D or 2.5D and how the context of topological abstraction influences the final result, depending on the operations applied to a certain set of data. Using a cell layout hierarchies are created and geometry can have a mesh representation. 2.3. Pedestrian Simulation Realistic crowd simulation involves a lot of components and variables such as group behavior, some kind of cognitive modeling, motion planning, rendering, trajectory computations and many others, the work by Guy [8] proposes an algorithm for simulating large-scale crowds which is based on the principle of least effort by using a mathematical model for representing the amount of effort that is expended by each agent. The work of Pelechano et al. [9] shows an approach on simulating the motion of realistic, large, dense crowds of autonomous characters moving in a natural manner and dynamically changing virtual environments. The work of [10] explains how Big Data management and analysis is vital in multiple scientific disciplines and how several areas of knowledge handle the different challenges that arise given a certain context. 3. Crowd Data and Simulation Simulating and populating large virtual environment is not a simple task, there are several problems that must be addressed and solved to properly display the crowd and achieve results that closely resemble reality. We present a system capable of handling several thousands of varied animated characters within a crowd. These characters are designed to have geometric, color animation and behavior variety, nevertheless when a crowd becomes bigger, more memory is needed and is often difficult to achieve this objective. We have designed this system taking into consideration several constraints, populating and simulating different scenarios in real-time using data that matches
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629
4
625
Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 2. A simplified XML that controls the distribution of elements inside the crowd.
reality, for example GPS data for the pedestrians to follow. The implementation of the system is further explained in the work of Toledo et al [11].
3.1. Populating the environment Constructing virtual environments is a task that can take a lot of time, effort and resources even for well established groups of animators and designers. Hand crafting every single member of the crowd or asset that is involved within the simulation is virtually impossible. We have developed an XML-based architecture that we use to create the general distribution of agents in the environment. Figure 3.1 shows a simplified version of what we use to parse the most important features of the crowd. By using this file we are able to describe the crowd using as many parameters as we want, for example we can determine how many elements will take part of the simulation and how many groups are involved, in this case we have a crowd composed by 5000 elements which are divided into different categories. In this particular case we determined to have a 40 per cent of male population of the global size of the crowd and that percentage is defined to have visual variety. We use this file to configure parameters not only for visualization but for simulation as well. For example we can go deeper and determine that a varied group of females can have a predominant age group or use a particular set of winter clothes and load the proper assets. But as mentioned previously, the file is not limited to visualization; we can extend the functionality of the system and determine how each agent will behave within the simulation, we can determine that a certain category simulation will be updated using social forces while other group might use GPS data, fuzzy logic or finite state machines just to state some cases. The system is robust enough to handle this kind of configurations as long as the proper shader is provided. Figure 3.1 shows the results of creating a urban environment by using the previously discussed techniques, by using the WRLD plug-in we collect the geographic information required, for our approach we consider two different techniques that we combine; first we collect data from GPS traces that describe routes that pedestrians take within the city this includes data such as latitude, longitude, elevation and a time stamp from when the sample was collected. From this moment on our agents can follow each of the routes and be animated accordingly, nevertheless it is important to address that memory access and storage can be an important bottleneck in the simulations, since ideally every agent should have one copy of the route that must follow, which makes the memory requirements grow significantly. To
626
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
5
Fig. 3. Results of creating the virtual environment.
Fig. 4. Instances of the elements of the crowd are created using dynamic materials.
avoid this we create GPS traces that work as guides to the agents and we create different instances at different points in time to mitigate the issue. Figure 3.1 shows how we integrate the previously discussed techniques and how the variety is achieved, compared to the work of Millan et al.[12] and Kavan et al. [13] we achieve the same frame rates without the memory constraints, for instance those approaches needed, roughly 12 megabytes for each character representation, animations and views, that means that a crowd composed by 1000 visually unique characters would need 12,000 megabytes of memory. We can achieve comparable results in terms of frame-rates without memory constraints, crowds of 10,000 agents can run using less than 300 megabytes of memory.
6
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629
627
Author name / Procedia Computer Science 00 (2018) 000–000
Fig. 5. A detailed description of all the data that is needed to render a single frame of the simulation.
3.2. Behavior and motion planning for pedestrians As mentioned previously, the way the agents move and react within the simulation is determined by mixing datadriven steering and group social forces. We use a database of preprocessed trajectories of pedestrians which we store as a set of vectors that encode the steering motion of pedestrians interacting with each other in public spaces. This data is used to generate steering actions given certain inputs, which can be a set of states that affect the surroundings of any virtual character. This set of actions is complemented using Helbing’s social group forces than might be found on real scenarios [14]. Nevertheless, pedestrian behavior is a very complex area of study that is not possible to model completely by using deterministic methods, at least to certain degree of realism. To solve this problem we incorporate fuzzy logic into the simulation, this way we are able to profile agents within the simulation, the system allows us to profile agents and assign values to a shared library of parameters that all characters inherit from. This way we can create two different profiles for all the elements in the simulation, the first is focused on the senses of each agent, such as vision range, maximum speed, weight, to state a few, and the second profile is oriented to the understanding of fuzzy concepts for each characters, for instance, agents may know what ”fast” or ”slow” are, but have completely different meaning regarding their physical capabilities. One of the main advantages of this method is that all agents have access to this knowledge and without major chances we can achieve a lot of variety in the crowd behavior.
4. Results For our experiments we executed the simulation and visualization process in a workstation with these characteristics: Intel Core i7-4810MQ CPU @ 2.80GHz 8, 16Gb of RAM, GeForce 880M with Gb of video memory. For a simulation in the city (Barcelona) with one thousand characters 4.3 million triangles are needed and 3.1 million vertices. We use 1115 draw calls and 39 batched draw calls. It takes a total of 300 megabytes of RAM memory, 112 megabytes of video memory and it takes 28 milliseconds to render each individual frame giving us a 35.71 frame rate. In average each frame has 18000 objects and the total scene is composed by 42767 objects. Figure 4 shows in detail every piece of data that is needed to render a single frame of the simulation, in this case the table is displaying the inputs needed to compute a mesh for any given zone in the urban environment, we do calculations using shaders in the GPU.
628
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
7
Fig. 6. Instances of the elements of the crowd are created using dynamic materials.
Figure 4, shows how the previously calculated mesh is transformed into an instance of the scene and elements are placed using both the geometric information of the system and the geographic coordinates that are associated with each of the elements in the simulation. 5. Conclusions and Future work As shown in this work, crowd simulation is a very complex area that involves many other disciplines, which have helped us to understand how to apply different techniques and handle date efficiently to simulate dense crowds in real-time Applications for virtual cities and complex environments range from research and educational purposes such as urban planning and simulation, to a wider area such as the movie and entertainment industry, security, traffic simulation, disaster planning and many more. With our approach, we have developed a robust system that can handle interactions within pedestrians and visualization of environments. As future work, it is important to extend the scope of the simulation and optimize the way we store and transfer our data to our agents, sometimes GPS traces can be heavy and slow to parse, and at this moment we are not able to process them in real time, meaning that we cannot simulate characters that use different routes that the ones we have already pre-computed. It would be interesting to launch the simulation using geo-spatial queries and adapt the level of detail algorithms we currently employ to further expand the simulation. Acknowledgements Special thanks to SECITI in Mexico for providing part of the founding for the present research, also we must express our gratitude for all the team that conforms the Barcelona Super Computing Center for facilitate the equipment and the infrastructure for the project. References [1] D. Thalmann, H. Grillon, J. Maim, B. Yersin, Challenges in Crowd Simulation, 2009 International Conference on CyberWorlds (2009) 1– 12doi:10.1109/CW.2009.23. URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5279720
8
Leonel Antonio Toledo Diaz et al. / Procedia Computer Science 139 (2018) 622–629 Author name / Procedia Computer Science 00 (2018) 000–000
629
[2] C. O’Sullivan, J. Cassell, H. Vilhjlmsson, J. Dingliana, S. Dobbyn, B. McNamee, C. Peters, T. Giang, Levels of detail for crowds and groups, Computer Graphics Forum 21 (4) (2002) 733–741. doi:10.1111/1467-8659.00631. URL http://dx.doi.org/10.1111/1467-8659.00631 [3] L. Toledo, O. De Gyves, I. Rivalcoba, I. Rudomn, Hierarchical Level of Detail for Varied Animated Crowds (2014). [4] P. van Oosterom, P. van Oosterom, S. Zlatanova, F. Penninga, E. Fendel, Advances in 3D Geoinformation Systems, 1st Edition, Springer Publishing Company, Incorporated, 2007. [5] R. van Essen, Maps Get Real: Digital Maps evolving from mathematical line graphs to virtual reality models, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008, pp. 3–18. [6] A. Slingsby, J. Raper, Navigable Space in 3D City Models for Pedestrians, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008, pp. 49–64. [7] A. Thomsen, M. Breunig, E. Butwilowski, B. Broscheit, Modelling and Managing Topology in 3D Geoinformation Systems, Springer Berlin Heidelberg, Berlin, Heidelberg, 2008, pp. 229–246. [8] S. J. Guy, J. Chhugani, S. Curtis, P. Dubey, M. Lin, D. Manocha, Pledestrians: A least-effort approach to crowd simulation, in: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’10, Eurographics Association, Goslar Germany, Germany, 2010, pp. 119–128. URL http://dl.acm.org/citation.cfm?id=1921427.1921446 [9] N. Pelechano, J. M. Allbeck, N. I. Badler, Controlling individual agents in high-density crowd simulation, in: Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA ’07, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 2007, pp. 99–108. URL http://dl.acm.org/citation.cfm?id=1272690.1272705 [10] C. L. P. Chen, C.-Y. Zhang, Data-intensive applications, challenges, techniques and technologies: A survey on big data, Inf. Sci. 275 (2014) 314–347. [11] L. Toledo, I. Rivalcoba, I. Rudom´ın, Fuzzy and data-driven urban crowds, in: Computational Science - ICCS 2018 - 18th International Conference, Wuxi, China, June 11-13, 2018 Proceedings, Part III, 2018, pp. 280–290. doi:10.1007/978-3-319-93713-7\_23. URL https://doi.org/10.1007/978-3-319-93713-7_23 [12] E. Millan, I. Rudomin, Impostors and pseudo-instancing for gpu crowd rendering, in: Proceedings of the 4th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, GRAPHITE ’06, ACM, New York, NY, USA, 2006, pp. 49–55. doi:10.1145/1174429.1174436. URL http://doi.acm.org/10.1145/1174429.1174436 ˇ ara, C. O’Sullivan, Polypostors: 2d polygonal impostors for 3d crowds, in: Proceedings of the 2008 [13] L. Kavan, S. Dobbyn, S. Collins, J. Z´ Symposium on Interactive 3D Graphics and Games, I3D ’08, ACM, New York, NY, USA, 2008, pp. 149–155. doi:10.1145/1342250. 1342273. URL http://doi.acm.org/10.1145/1342250.1342273 [14] M. Moussa¨ıd, N. Perozo, S. Garnier, D. Helbing, G. Theraulaz, The walking behaviour of pedestrian social groups and its impact on crowd dynamics., PloS one 5 (4) (2010) e10047. doi:10.1371/journal.pone.0010047. URL http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2850937