- Email: [email protected]
Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation
Transportation Research Part F xxx (2017) xxx–xxx
Contents lists available at ScienceDirect
Transportation Research Part F journal homepage: www.elsevier.com/locate/trf
Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation Andreas Richter ⇑, Michael Scholz German Aerospace Center, Institute of Transportation Systems, Lilienthalplatz 7, 38108 Braunschweig, Germany
a r t i c l e
i n f o
Article history: Received 29 December 2016 Accepted 10 April 2017 Available online xxxx Keywords: Discrete geodata Surveying Guidelines Intermediate data format Driving simulation Automated driving
a b s t r a c t Development of driver assistance and automation systems relies on domain-specific formats for the geometrical and logical representation of road networks in simulation environments. The trend to simulate real world urban environments leads to increasing demands for such data which cannot be derived easily from cadastral or open source geodata. In contrast, specific surveying directly into these domain-specific formats quickly becomes time- and cost-consuming. The DLR in partnership with OEMs developed guidelines and a simplified data format to facilitate surveying into an intermediate, discrete geodata format which meets the requirements of both the governmental and the driving simulation domain. The intermediate format should serve as a data hub from which specific simulation formats can be derived automatically through the developed processing tool chain. In this article the need of such an approach is discussed, the data format and the guidelines explained as well as the feasibility and effort of this approach is examined in an urban use case in Germany covering the dedicated surveying of road sections followed by automatic processing into OpenDRIVE. Ó 2017 Elsevier Ltd. All rights reserved.
1. Introduction Driver assistance and automation systems are becoming increasingly complex. Subsequently, the validation and verification of these systems is also increasing in complexity. Therefore simulation is an important part of developing and testing of these systems to mitigate rising cost and time efforts (Richter, Fischer, Frankiewicz, Schnieder, & Köster, 2015). The simulation requires road networks to be modelled, which is very time-consuming when done manually especially if complex urban databases are needed. It is obvious to use pre-existing geodata and process them with the help of an automated tool chain approach to avoid manual work. The project ‘‘Virtual World” already demonstrated possibilities and challenges in doing that (Richter, Scholz, Friedl, Ruppert, & Köster, 2016). One conclusion is that there is no perfect data source to provide all necessary information in the accuracy and completeness as needed. Publicly available and cadastral geodata is often difficult to use for modelling, in the context of simulation, because it lacks necessary resolution and metadata. The representation of road features in said datasets is incomplete to the specific level of detail required by driving simulators. Projects dealing with the extraction of additional road information from remote sensing (Mattyus, Wang, Fidler, & Urtasun, 2015) show that the derived data lacks necessary precision needed for testing and automation use cases. Specific road surveying is a good ⇑ Corresponding author. E-mail addresses: [email protected] (A. Richter), [email protected] (M. Scholz). http://dx.doi.org/10.1016/j.trf.2017.04.004 1369-8478/Ó 2017 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
2
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
solution for these problems because it already gathers the data from the subject’s perspective. The limiting factor for road surveying is that it can become complex, time-consuming and expensive, because the raw data is pre-processed manually. Often for new driving simulator environments data may be gathered extensively by mobile mapping but transformed manually (Boer, Penna, Utz, Pedersen, & Sierhuis, 2015) and stored in a proprietary data format (Khastgir, Birrell, Dhadyalla, Fulker, & Jennings, 2015). Not only the development of driver assistance and automation systems needs detailed road networks to create simulator databases but also productive use of these systems later depends on lane-detailed maps as input. Thus the question about appropriate and detailed geodata arises again. Once more public data does not deliver all necessary information in required accuracy and road surveying is getting expensive. Guidelines for surveying and modelling and a simplified data model should solve this issue. The goal is to combine the availability of official large-area data from geographic information systems domain (GIS) and CAD data with additional details to be used for simulator databases and as highly precise maps for automated driving. The simplified data model serves as a hub for automated conversion of the geodata into domain-specific data formats and can still be used independently in GIS applications as well. Following the guidelines reduces the effort of pre-processing the raw data into that simplified data format. Also already existing official data can be converted using these guidelines. 1.1. Available data For some years now official geodata gets extensively digitised, for example through the INSPIRE initiative (Infrastructure for Spatial Information in the European Community), which has a strong focus on cadastral data (European Commission, 2016). Public authorities are using these data for the generation of digital city models, for solar potential analysis and for computation of noise propagation. Information about infrastructure and urban green is managed in a digital way and aerial surveys generate not only images but also derived digital city models. The quality and availability of data is very heterogeneous when comparing different local authorities. It depends on how many resources can be used for maintaining the GIS. In Braunschweig, Germany, e.g. the road infrastructure is managed in a spatial database with attributes such as (hardware) type, orientation, etc. In comparison to that in Stuttgart, Germany, a georeferenced CAD tile system is used. In those CAD tiles road signs are stored as drawn graphics without the possibility to extract discrete digital infrastructure databases easily nor automatically. The automotive domain is already using geodata as navigation data for decades. These navigation databases are modelled as graphs with additional helpful information such as number of lanes and points of interest. But this information is not modelled in a precise way because it is used only for visualisation purpose and because of the inaccuracy of the GNSS position. Specialised data contains information about the elevation profile or detailed course of the road to support drivetrain applications. The current development of this data is focussing on lane-level modelling (NDS Association, 2016). This ‘‘lane product” focuses only on road lanes and boundaries, pedestrian walks, etc. are not considered. Also currently only low coverage of road networks is realised with the new data formats, therefore it can be used only for testing but not for large scale usage. Google for instance is generating their maps for automated driving manually using the vehicles sensor data (Google Self-Driving Car Project. Monthly Report March, 2016). Crowd-sourced data such as OpenStreetMap (OSM) should be mentioned as well. Especially in Germany the OSM community is very vivid and maps cities very ambitiously. They include also valuable data about the road network such as number of lanes, road restrictions and infrastructure. Unfortunately the limited accuracy and representation of the collected data is unsuitable for automated driving purposes and for simulator applications (Richter, Friedl, & Scholz, 2016). 1.2. Used data formats The process of transferring road networks from reality to simulation has many gaps to fill (Friedl & Richter, 2012). In the context of simulation, driving simulators are using road description formats, which are application- and domain-specific to the context of automotive simulation. These formats are implementing mathematical representations of geometric road features with strictly logical relationships and relative attribution. Examples of such domain-specific simulation formats are implemented by OpenDRIVE, RoadXML and IPGRoad. In contrast, public cadastral geodata historically follows the concept of pure visual representation of real-world features. An example being road surfaces represented simply by their discrete boundary lines or as strongly generalised polygons without direct relationship to punctual traffic infrastructure elements. Often this data is derived from analogue maps and therefore contains heterogeneous accuracy due to sloppy digitalisation. Transforming the latter format into domainspecific simulation formats require the synthesis of additional geometrical information such as lane boundaries and logical relationships between road and infrastructure features. Traffic flow simulations and also navigation applications work on routable road networks, such as Open Street Map (OSM), which are mostly graph-based and can be used to derive logical information. They are provided in GPS Exchange Format (GPX) or as binary stored databases. Pros and Cons about these data are discussed in the previous section. A combination of different data sources for driving and traffic simulation purposes is generally possible (Richter et al., 2016). To compensate for these constraints and to cope with the specific demands of simulation formats systematic surveying of geodata through mobile mapping is employed. Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
3
2. Data acquisition The acquisition of suitable data for simulation applications has to be divided into the task of surveying and the task of data transformation into the target formats. The data formats used in driving simulations and automated driving are highly sophisticated and are way beyond the complexity of cadastral data. Conventional stationary surveying or data acquisition through mobile mapping provides discrete cadastral data with characteristics as described in the previous section. Common surveying companies are not familiar with transferring and transforming their measured raw data into those domainspecific formats. OpenDRIVE and RoadXML require all road and infrastructure features to be described mathematically and relatively whereas Navigation Data Standard (NDS) is not accessible under public domain and is implemented through a binary database format (NDS Association, 2014). Transformation of survey data into the target formats quickly becomes costly because providers have to manually process and annotate their raw measurements. Urban road intersection areas with their complex logics are error-prone and result in exploding time consumption for editing. Little competition between providers is possible because each has to develop a workflow for derivation of the target format. In total, a specific surveying directly into the desired domain-specific format needs time above-average and can lead to unpredictable costs. Especially the upcoming usage of NDS for autonomous driving boosts the need of having surveyed data in such complex data formats. In the case of a governmental institution, which orders a conventional survey to build up or update a cadastre, the target format again is likely to be discrete, generalised and unsuitable for easy post-processing for the use in simulation domain applications. For this purpose we developed guidelines for the surveying of conventional geodata with extended attribution which can support easier and automated processing into above described simulation formats. 3. Developing of guidelines and a simplified data format Over the last few years the German Aerospace Center (DLR) and many OEMs have gained experience with the complexity and effort of data acquisition and transformation. Currently there is little reference available that tackles this task. All published work solves this challenge through manual work [e.g. 4] or with Google’s approach (Google Self-Driving Car Project, 2016). High-precision road data is not only used by driving simulations but is also valuable for authorities, road operators and necessary for automated driving. It should be possible to gather road data in a way that suites all purposes. The solution is a guide for road surveying and pre-processing of the raw data while adapting the requirements of potential users. Following the guidelines and given modelling examples should facilitate to provide the surveyed data in a simplified format which can be transformed into additional sophisticated formats subsequently. It should not be used as ‘‘yet another standardised format” but serve as an intermediate data hub. This intermediate format can allow conversion between different specific formats without implementing direct transformation. Therefore specialised information can be stored in the intermediate data format and is not lost after transformation. The intermediate format is using OGC simple features to be visualised with custom GIS as well. The mathematical approximation and transformation can be decoupled from each conversion step and be adjusted individually. Each
Fig. 1. Schematic representation of the conversion process.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
4
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
conversion module can be updated independently form other modules without having impact on their implementation. The schema of the approach of using an intermediate data format is given in Fig. 1. To deploy such a solution the DLR in partnership with Audi, BMW, Daimler, Porsche and Volkswagen started the project ‘‘Road2Simulation” to analyse the challenges in detail. As a result of this a simplification and standardisation of the process is proposed and summarised in guidelines combined with a simplified data format (Richter & Scholz, 2016), both freely available. These guidelines meet the requirements from the simulator formats keeping automated driving in mind as well as from cadastral data. The project started with the focus on German roads and will be extended by new user requirements. 3.1. Key elements of the guidelines Basic idea of the guidelines is a simple surveying through external (mobile mapping) providers in connection with automatic conversion by the end user, respectively in the simulation domain. With the help of an intermediate and simplified data model the conversion should be facilitated. This simplified model is attributed with additional metadata, which is needed for generation of the road network and derived from the specification of the target format. The intermediate format was aligned to the OpenDRIVE format (Dupuis & et al., 2015) because it is a de facto standard in driving simulator domain and used by all Road2Simulation project partners. Additionally OpenDRIVE is royalty-free and other formats have a close resemblance to this format. Fig. 2 shows the key elements of the simplified data model and their relation. Basis of the guidelines are polylines with a heterogeneous density of specified points for description of the road course. A high density of sampling points is used for an accurate modelling, e.g. in curves, and low density is usable for straight sections. This heterogeneous density supports high-precision as well as simplified modelling if this data is interpolated during transformation into the domain-specific format. Further, lanes and their lane marks, barriers and other linear objects build up the main network. Punctual coordinates describing infrastructure elements such as road signs, traffic lights, special markings on the road and street lights or
Fig. 2. Simplified data model of the guidelines with key elements (blue) and hierarchical elements (grey). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
5
important street furniture, are also part of the model. Some elements like traffic islands are currently difficult to model in mentioned simulator formats. At the time of writing they are covered as areas (polygons), thus no information is lost in the intermediate data format. All features modelled are using 3D simple features stored as well-known binary or wellknown text based in OGC Simple Feature Access (OGC Inc., 2011). This supports the accessibility by common GIS tools. All these elements are enriched with additional metadata such as information about parental elements, properties that describe the type and appearance of the element and a link to a description of the data source. For example, a lane border element contains – beside of a parent, a geometry and data source information – the type of its outlining lane. It can define a driving or parking lane, a restricted lane or sidewalks, etc. The used building material can be asphalt, concrete, pavement or cobble, etc. The data source description of each element consists of information about the absolute and relative error, date and a quality description. The quality description includes a natural description about the data source, for example name of a cadastre file including its creation date or a sensor including its firmware version, and about the kind of processing of raw, cleaned, post-processed or fused data. With this information users should be able to assess the data for their specific needs. 3.2. Examples and suggestions Guidelines will not be easily accessible if they do not provide examples and give suggestions for specific situations. Therefore, for all common road situations in Germany such as one- and multi-lane roads, one-way roads, motorways, roads with physical separation, complex intersections and roundabouts, examples are prepared in the guidelines. They explain which features, details and information have to be gathered and selected step by step (Richter et al., 2016). The core elements should be modelled in a specific way to support a sufficient data transformation. For example, the course of a road is always modelled as connection between two intersections; such road elements are never attached directly to each other. The polyline describing the course of the road should be modelled as polyline in the centre of the road if there is no separation of the road. In the case of physical separation the course polyline should be modelled on the left side of the leftmost lane in driving direction for right-handed traffic. Some examples of different road situations are shown in Fig. 3. The lane borders are describing the appearance of the road not only by their attributes (type and material) but especially with the help of the course of the polyline. Derivation in the height can be used to model dropped kerbs. Lateral derivation can precisely model outer edges of tarmac areas if necessary. In Fig. 3 in the bottom left image a parking space is modelled in that manner.
Fig. 3. Different road situations with marked course of the road (green) and lane borders (cyan). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
6
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
Especially intersections with their lane connections need special treatment because often not all technically necessary construction paths are present in reality as they are needed for simulator formats. Fig. 4 shows an intersection with connections in all cardinal directions. Additionally U-turn connections are modelled, which might not be available in reality but necessary to model the appearance of the physical separation. Complex intersections are exemplarily decompiled in less complex and smaller intersections like shown in Fig. 5. This approach is also used for roundabouts as well as highway entries and exits (as shown in Fig. 3 in the bottom right). Punctual information about road infrastructure and furniture are rather a chapter apart. To represent this infrastructure logically only location, relation and type is necessary but for visualisation a lot more details such as appearance, text, orientation, etc. are needed. Fig. 6 shows the amount of punctual information for only one connection direction. To adapt the effort of pre-processing according to the requirements the guidelines propose a standard way of modelling and an extended way of modelling. In the standard way only information about the road and crucially necessary infrastructure is modelled. Thus the representation ends right after the kerbside like the NDS format describes it. The extended way includes everything else, giving the whole traffic area its appearance. It could be enhanced or reduced depending of the target format. As conclusion, the guidelines for surveying of a simplified, intermediate road model suffice the demands from both cadastral authorities and complex simulation formats, as the latter can be obtained through our developed processing tool chain. In the end it does not necessarily matter if the simplified model is created from mobile mapping, derived from aerial imagery or from cadastral data. The approach includes a metadata catalogue covering information about geometric accuracy, technical details about the acquisition procedure and, if applicable, already applied processing steps. 4. Usage of the guidelines The guidelines were used in the project ‘‘Road2Simulation” for a prototypical road survey. A 14 km long test track in Stuttgart, Germany, was selected consisting of urban, non-urban, tunnel and motorway sections. The road survey was conducted in Q2 2016 and the outcome was available in Q4 2016. The resulting road network will be provided in OpenDRIVE and OpenCRG format (which is also included in the guidelines), a 3D model with the corresponding environment for driving simulator usage is generated in OpenSceneGraph format based on different cadastral data sources and using the ‘‘Virtual World” approach (Richter et al., 2016). Fig. 7 shows the pre-processed survey data compared in a GIS visualisation with the cadastral data (e.g. digital terrain model as well as digital city model and tree cadastre) of the city of Stuttgart. The surveying contractor was selected in a two-staged process where all applicants had to qualify by applying the guidelines to a sample intersection. We received ten replies for our request from different survey companies. After that six qualified contractors were called for an offer to survey the test track according to the guidelines and deliver a section of surface data, too. The bids ran one third of comparable bids requesting for direct delivery in domain-specific simulator formats.
Fig. 4. Intersection with connections for each cardinal direction.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
7
Fig. 5. Decompiled complex intersection.
Fig. 6. Punctual information (magenta) about signage, street lighting and road furniture. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Fig. 7. Surveyed and pre-processed data in combination with cadastral data.
The contractor was able to adopt the guidelines easily without crucial misunderstanding. In the beginning there was a loss of time because of a homogenisation issue of the laser scanning raw data but the main work was done within a timeframe of two months despite the huge amount of elements that had to be considered. Nevertheless some problems occurred due to sloppy work but could be fixed with little effort because no complex mathematical modelling had to be reworked.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
8
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
After that the tool chain for automated transformation into OpenDRIVE had to be improved to comply better with real-life examples. 5. Outlook The guidelines are currently updated after validation with the help of the survey result and by gathered experience. Afterwards they will be published for free again in order for other companies and authorities to use and adapt them for costefficient gathering of road data. This should result in a win-win-situation because the additionally surveyed attributes help governmental data, for example, to become interesting for third parties, especially for the automotive domain. It should also boost the open data movement by providing more machine-processable cadastral data and support the possibility to reduce the dependency on specialised data providers to survey and manage large-scale road network data. The guidelines and the simplified data format will be used for larger-scale surveying in upcoming test sites and corresponding projects that need high-precision data for development and test of automated driving applications. Additionally it is being planned to publish the tool chain for transforming the simplified data into OpenDRIVE and/or NDS, too. Later on the guidelines can be extended for additional description formats as well as for further road situations such as parking spaces or specific conditions in other countries. Also the need of describing the environment next to the road is arising. Buildings, street vegetation and the terrain should also be modelled. Thus, kind of a ‘‘general world description format” is necessary. It will be challenging to include all details and specialities in a feasible way to represent e.g. buildings realistically. 6. Conclusion The need for real-world data to test advanced assistance and automation system is growing fast. Currently the acquisition of ready-to-use real-world data for driving simulator applications is expensive. There is no adoptable approach available to minimise these efforts. Thus, a solution for reducing the complexity of surveying and transforming the geodata was developed by DLR in partnership with OEMs. Key element of this solution is an intermediate, discrete geodata format, which includes necessary metadata for transformation. The data model together with modelling suggestions and examples for typical road situations were combined into guidelines and made available free of charge. The developed guidelines are tested with a prototypical survey and data transformation and will be updated for new formats and additional road situations. Funding This work based on the experience and work of DLR that was funded by the Helmholtz Association. The development of the guidelines and simplified data model was funded by Audi, BMW, Daimler, Porsche and Volkswagen. Acknowledgements We would like to thank Hartmut Friedl and Thomas Ruppert from DLR’s Remote Sensing Data Center, Oberpfaffenhofen, Germany, for the support in dealing with the cadastral data. References Boer, E. R., Penna, M. D., Utz, H., Pedersen, L., Sierhuis, M. (2015). The role of driving simulators in developing and evaluating autonomous vehicles. In Driving Simulator Conference 2015 Europe Germany, 16–18 Sep. 2015, Tübingen, Germany. ISBN 978-3-9813099-3-5. Dupuis, M., et al., (2015). OpenDRIVEÒ Format Specification, Rev. 1.4 Issue H.. Accessed 22. Dec. 2016. European Commission (2016). About INSPIRE. Accessed 22. Dec. 2016 . Friedl, H., & Richter, A. (2012). Fusion heterogener Geodaten zur Erstellung realer 3D-Welten am Beispiel einer Fahrsimulation. GEOINFORMATIK , 28–30 March 2012. Germany: Braunschweig. ISBN 978-3-8440-0888-3. ISSN 1618-1034. Google Self-Driving Car Project. Monthly Report March 2016, . Accessed 22. Dec. 2016. Khastgir, S., Birrell, S., Dhadyalla, G., Fulker, D., Jennings, P. (2015). A drive-in, driver-in-the-loop simulator for testing autonomous vehicles. In Driving simulator conference 2015 Europe Germany, 16–18 Sep. 2015, Tübingen, Germany. ISBN 978-3-9813099-3-5. Mattyus, G., Wang, S., Fidler, S., Urtasun, R. (2015). Enhancing road maps by parsing aerial images aroand the world. Computer Vision (ICCV). In IEEE international conference on computer vision, 13–16 Dec. 2015, Santiago de Chile, Chile. NDS Association (2014). On the way – NDS success stories from 2014. Accessed 22. Dec. 2016 . NDS Association, 2016. NDS Open Lane Model 1.0 Release. . Accessed 22. Dec. 2016. OGC Inc. (2011) OpenGISÒ Implementation standard for geographic information - Simple feature access - Part 1: Common architecture. Version 1.2.1. http:// www.opengeospatial.org/standards/sfa. Accessed 22. Dec. 2016. Document reference number: OGC 06-103r4. Richter, A., Fischer, M., Frankiewicz, T., Schnieder, L., Köster, F. (2015). Reducing the gap between simulated and real life environments by introducing highprecision data. In Driving simulator conference 2015 Europe Germany, 16–18 Sep. 2015, Tübingen, Germany. ISBN 978-3-9813099-3-5. Richter, A., Scholz, M. (2016). Road2Simulation guidelines – Leitfaden zur Erhebung von Straßendaten für Simulation und Entwicklung. . Accessed 22. Dec. 2016.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
9
Richter, A., Scholz, M., Friedl, H., Ruppert, T., Köster, F. (2016). Challenges and experiences in using heterogeneous, geo-referenced data for automatic creation of driving simulator environments. Simulation-transactions of the society for modeling and simulation international, 92(5), 437–446. SAGE Publications. http://dx.doi.org/10.1177/0037549716641201. ISSN 0037-5497. Richter, A., Friedl, H., Scholz, M. (2016). Beyond OSM – alternative data sources and approaches enhancing generation of road networks for traffic and driving simulations. In SUMO2016 – Traffic, Mobility, and Logistics. 23–25 May 2016, Berlin, Germany.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
Contents lists available at ScienceDirect
Transportation Research Part F journal homepage: www.elsevier.com/locate/trf
Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation Andreas Richter ⇑, Michael Scholz German Aerospace Center, Institute of Transportation Systems, Lilienthalplatz 7, 38108 Braunschweig, Germany
a r t i c l e
i n f o
Article history: Received 29 December 2016 Accepted 10 April 2017 Available online xxxx Keywords: Discrete geodata Surveying Guidelines Intermediate data format Driving simulation Automated driving
a b s t r a c t Development of driver assistance and automation systems relies on domain-specific formats for the geometrical and logical representation of road networks in simulation environments. The trend to simulate real world urban environments leads to increasing demands for such data which cannot be derived easily from cadastral or open source geodata. In contrast, specific surveying directly into these domain-specific formats quickly becomes time- and cost-consuming. The DLR in partnership with OEMs developed guidelines and a simplified data format to facilitate surveying into an intermediate, discrete geodata format which meets the requirements of both the governmental and the driving simulation domain. The intermediate format should serve as a data hub from which specific simulation formats can be derived automatically through the developed processing tool chain. In this article the need of such an approach is discussed, the data format and the guidelines explained as well as the feasibility and effort of this approach is examined in an urban use case in Germany covering the dedicated surveying of road sections followed by automatic processing into OpenDRIVE. Ó 2017 Elsevier Ltd. All rights reserved.
1. Introduction Driver assistance and automation systems are becoming increasingly complex. Subsequently, the validation and verification of these systems is also increasing in complexity. Therefore simulation is an important part of developing and testing of these systems to mitigate rising cost and time efforts (Richter, Fischer, Frankiewicz, Schnieder, & Köster, 2015). The simulation requires road networks to be modelled, which is very time-consuming when done manually especially if complex urban databases are needed. It is obvious to use pre-existing geodata and process them with the help of an automated tool chain approach to avoid manual work. The project ‘‘Virtual World” already demonstrated possibilities and challenges in doing that (Richter, Scholz, Friedl, Ruppert, & Köster, 2016). One conclusion is that there is no perfect data source to provide all necessary information in the accuracy and completeness as needed. Publicly available and cadastral geodata is often difficult to use for modelling, in the context of simulation, because it lacks necessary resolution and metadata. The representation of road features in said datasets is incomplete to the specific level of detail required by driving simulators. Projects dealing with the extraction of additional road information from remote sensing (Mattyus, Wang, Fidler, & Urtasun, 2015) show that the derived data lacks necessary precision needed for testing and automation use cases. Specific road surveying is a good ⇑ Corresponding author. E-mail addresses: [email protected] (A. Richter), [email protected] (M. Scholz). http://dx.doi.org/10.1016/j.trf.2017.04.004 1369-8478/Ó 2017 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
2
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
solution for these problems because it already gathers the data from the subject’s perspective. The limiting factor for road surveying is that it can become complex, time-consuming and expensive, because the raw data is pre-processed manually. Often for new driving simulator environments data may be gathered extensively by mobile mapping but transformed manually (Boer, Penna, Utz, Pedersen, & Sierhuis, 2015) and stored in a proprietary data format (Khastgir, Birrell, Dhadyalla, Fulker, & Jennings, 2015). Not only the development of driver assistance and automation systems needs detailed road networks to create simulator databases but also productive use of these systems later depends on lane-detailed maps as input. Thus the question about appropriate and detailed geodata arises again. Once more public data does not deliver all necessary information in required accuracy and road surveying is getting expensive. Guidelines for surveying and modelling and a simplified data model should solve this issue. The goal is to combine the availability of official large-area data from geographic information systems domain (GIS) and CAD data with additional details to be used for simulator databases and as highly precise maps for automated driving. The simplified data model serves as a hub for automated conversion of the geodata into domain-specific data formats and can still be used independently in GIS applications as well. Following the guidelines reduces the effort of pre-processing the raw data into that simplified data format. Also already existing official data can be converted using these guidelines. 1.1. Available data For some years now official geodata gets extensively digitised, for example through the INSPIRE initiative (Infrastructure for Spatial Information in the European Community), which has a strong focus on cadastral data (European Commission, 2016). Public authorities are using these data for the generation of digital city models, for solar potential analysis and for computation of noise propagation. Information about infrastructure and urban green is managed in a digital way and aerial surveys generate not only images but also derived digital city models. The quality and availability of data is very heterogeneous when comparing different local authorities. It depends on how many resources can be used for maintaining the GIS. In Braunschweig, Germany, e.g. the road infrastructure is managed in a spatial database with attributes such as (hardware) type, orientation, etc. In comparison to that in Stuttgart, Germany, a georeferenced CAD tile system is used. In those CAD tiles road signs are stored as drawn graphics without the possibility to extract discrete digital infrastructure databases easily nor automatically. The automotive domain is already using geodata as navigation data for decades. These navigation databases are modelled as graphs with additional helpful information such as number of lanes and points of interest. But this information is not modelled in a precise way because it is used only for visualisation purpose and because of the inaccuracy of the GNSS position. Specialised data contains information about the elevation profile or detailed course of the road to support drivetrain applications. The current development of this data is focussing on lane-level modelling (NDS Association, 2016). This ‘‘lane product” focuses only on road lanes and boundaries, pedestrian walks, etc. are not considered. Also currently only low coverage of road networks is realised with the new data formats, therefore it can be used only for testing but not for large scale usage. Google for instance is generating their maps for automated driving manually using the vehicles sensor data (Google Self-Driving Car Project. Monthly Report March, 2016). Crowd-sourced data such as OpenStreetMap (OSM) should be mentioned as well. Especially in Germany the OSM community is very vivid and maps cities very ambitiously. They include also valuable data about the road network such as number of lanes, road restrictions and infrastructure. Unfortunately the limited accuracy and representation of the collected data is unsuitable for automated driving purposes and for simulator applications (Richter, Friedl, & Scholz, 2016). 1.2. Used data formats The process of transferring road networks from reality to simulation has many gaps to fill (Friedl & Richter, 2012). In the context of simulation, driving simulators are using road description formats, which are application- and domain-specific to the context of automotive simulation. These formats are implementing mathematical representations of geometric road features with strictly logical relationships and relative attribution. Examples of such domain-specific simulation formats are implemented by OpenDRIVE, RoadXML and IPGRoad. In contrast, public cadastral geodata historically follows the concept of pure visual representation of real-world features. An example being road surfaces represented simply by their discrete boundary lines or as strongly generalised polygons without direct relationship to punctual traffic infrastructure elements. Often this data is derived from analogue maps and therefore contains heterogeneous accuracy due to sloppy digitalisation. Transforming the latter format into domainspecific simulation formats require the synthesis of additional geometrical information such as lane boundaries and logical relationships between road and infrastructure features. Traffic flow simulations and also navigation applications work on routable road networks, such as Open Street Map (OSM), which are mostly graph-based and can be used to derive logical information. They are provided in GPS Exchange Format (GPX) or as binary stored databases. Pros and Cons about these data are discussed in the previous section. A combination of different data sources for driving and traffic simulation purposes is generally possible (Richter et al., 2016). To compensate for these constraints and to cope with the specific demands of simulation formats systematic surveying of geodata through mobile mapping is employed. Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
3
2. Data acquisition The acquisition of suitable data for simulation applications has to be divided into the task of surveying and the task of data transformation into the target formats. The data formats used in driving simulations and automated driving are highly sophisticated and are way beyond the complexity of cadastral data. Conventional stationary surveying or data acquisition through mobile mapping provides discrete cadastral data with characteristics as described in the previous section. Common surveying companies are not familiar with transferring and transforming their measured raw data into those domainspecific formats. OpenDRIVE and RoadXML require all road and infrastructure features to be described mathematically and relatively whereas Navigation Data Standard (NDS) is not accessible under public domain and is implemented through a binary database format (NDS Association, 2014). Transformation of survey data into the target formats quickly becomes costly because providers have to manually process and annotate their raw measurements. Urban road intersection areas with their complex logics are error-prone and result in exploding time consumption for editing. Little competition between providers is possible because each has to develop a workflow for derivation of the target format. In total, a specific surveying directly into the desired domain-specific format needs time above-average and can lead to unpredictable costs. Especially the upcoming usage of NDS for autonomous driving boosts the need of having surveyed data in such complex data formats. In the case of a governmental institution, which orders a conventional survey to build up or update a cadastre, the target format again is likely to be discrete, generalised and unsuitable for easy post-processing for the use in simulation domain applications. For this purpose we developed guidelines for the surveying of conventional geodata with extended attribution which can support easier and automated processing into above described simulation formats. 3. Developing of guidelines and a simplified data format Over the last few years the German Aerospace Center (DLR) and many OEMs have gained experience with the complexity and effort of data acquisition and transformation. Currently there is little reference available that tackles this task. All published work solves this challenge through manual work [e.g. 4] or with Google’s approach (Google Self-Driving Car Project, 2016). High-precision road data is not only used by driving simulations but is also valuable for authorities, road operators and necessary for automated driving. It should be possible to gather road data in a way that suites all purposes. The solution is a guide for road surveying and pre-processing of the raw data while adapting the requirements of potential users. Following the guidelines and given modelling examples should facilitate to provide the surveyed data in a simplified format which can be transformed into additional sophisticated formats subsequently. It should not be used as ‘‘yet another standardised format” but serve as an intermediate data hub. This intermediate format can allow conversion between different specific formats without implementing direct transformation. Therefore specialised information can be stored in the intermediate data format and is not lost after transformation. The intermediate format is using OGC simple features to be visualised with custom GIS as well. The mathematical approximation and transformation can be decoupled from each conversion step and be adjusted individually. Each
Fig. 1. Schematic representation of the conversion process.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
4
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
conversion module can be updated independently form other modules without having impact on their implementation. The schema of the approach of using an intermediate data format is given in Fig. 1. To deploy such a solution the DLR in partnership with Audi, BMW, Daimler, Porsche and Volkswagen started the project ‘‘Road2Simulation” to analyse the challenges in detail. As a result of this a simplification and standardisation of the process is proposed and summarised in guidelines combined with a simplified data format (Richter & Scholz, 2016), both freely available. These guidelines meet the requirements from the simulator formats keeping automated driving in mind as well as from cadastral data. The project started with the focus on German roads and will be extended by new user requirements. 3.1. Key elements of the guidelines Basic idea of the guidelines is a simple surveying through external (mobile mapping) providers in connection with automatic conversion by the end user, respectively in the simulation domain. With the help of an intermediate and simplified data model the conversion should be facilitated. This simplified model is attributed with additional metadata, which is needed for generation of the road network and derived from the specification of the target format. The intermediate format was aligned to the OpenDRIVE format (Dupuis & et al., 2015) because it is a de facto standard in driving simulator domain and used by all Road2Simulation project partners. Additionally OpenDRIVE is royalty-free and other formats have a close resemblance to this format. Fig. 2 shows the key elements of the simplified data model and their relation. Basis of the guidelines are polylines with a heterogeneous density of specified points for description of the road course. A high density of sampling points is used for an accurate modelling, e.g. in curves, and low density is usable for straight sections. This heterogeneous density supports high-precision as well as simplified modelling if this data is interpolated during transformation into the domain-specific format. Further, lanes and their lane marks, barriers and other linear objects build up the main network. Punctual coordinates describing infrastructure elements such as road signs, traffic lights, special markings on the road and street lights or
Fig. 2. Simplified data model of the guidelines with key elements (blue) and hierarchical elements (grey). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
5
important street furniture, are also part of the model. Some elements like traffic islands are currently difficult to model in mentioned simulator formats. At the time of writing they are covered as areas (polygons), thus no information is lost in the intermediate data format. All features modelled are using 3D simple features stored as well-known binary or wellknown text based in OGC Simple Feature Access (OGC Inc., 2011). This supports the accessibility by common GIS tools. All these elements are enriched with additional metadata such as information about parental elements, properties that describe the type and appearance of the element and a link to a description of the data source. For example, a lane border element contains – beside of a parent, a geometry and data source information – the type of its outlining lane. It can define a driving or parking lane, a restricted lane or sidewalks, etc. The used building material can be asphalt, concrete, pavement or cobble, etc. The data source description of each element consists of information about the absolute and relative error, date and a quality description. The quality description includes a natural description about the data source, for example name of a cadastre file including its creation date or a sensor including its firmware version, and about the kind of processing of raw, cleaned, post-processed or fused data. With this information users should be able to assess the data for their specific needs. 3.2. Examples and suggestions Guidelines will not be easily accessible if they do not provide examples and give suggestions for specific situations. Therefore, for all common road situations in Germany such as one- and multi-lane roads, one-way roads, motorways, roads with physical separation, complex intersections and roundabouts, examples are prepared in the guidelines. They explain which features, details and information have to be gathered and selected step by step (Richter et al., 2016). The core elements should be modelled in a specific way to support a sufficient data transformation. For example, the course of a road is always modelled as connection between two intersections; such road elements are never attached directly to each other. The polyline describing the course of the road should be modelled as polyline in the centre of the road if there is no separation of the road. In the case of physical separation the course polyline should be modelled on the left side of the leftmost lane in driving direction for right-handed traffic. Some examples of different road situations are shown in Fig. 3. The lane borders are describing the appearance of the road not only by their attributes (type and material) but especially with the help of the course of the polyline. Derivation in the height can be used to model dropped kerbs. Lateral derivation can precisely model outer edges of tarmac areas if necessary. In Fig. 3 in the bottom left image a parking space is modelled in that manner.
Fig. 3. Different road situations with marked course of the road (green) and lane borders (cyan). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
6
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
Especially intersections with their lane connections need special treatment because often not all technically necessary construction paths are present in reality as they are needed for simulator formats. Fig. 4 shows an intersection with connections in all cardinal directions. Additionally U-turn connections are modelled, which might not be available in reality but necessary to model the appearance of the physical separation. Complex intersections are exemplarily decompiled in less complex and smaller intersections like shown in Fig. 5. This approach is also used for roundabouts as well as highway entries and exits (as shown in Fig. 3 in the bottom right). Punctual information about road infrastructure and furniture are rather a chapter apart. To represent this infrastructure logically only location, relation and type is necessary but for visualisation a lot more details such as appearance, text, orientation, etc. are needed. Fig. 6 shows the amount of punctual information for only one connection direction. To adapt the effort of pre-processing according to the requirements the guidelines propose a standard way of modelling and an extended way of modelling. In the standard way only information about the road and crucially necessary infrastructure is modelled. Thus the representation ends right after the kerbside like the NDS format describes it. The extended way includes everything else, giving the whole traffic area its appearance. It could be enhanced or reduced depending of the target format. As conclusion, the guidelines for surveying of a simplified, intermediate road model suffice the demands from both cadastral authorities and complex simulation formats, as the latter can be obtained through our developed processing tool chain. In the end it does not necessarily matter if the simplified model is created from mobile mapping, derived from aerial imagery or from cadastral data. The approach includes a metadata catalogue covering information about geometric accuracy, technical details about the acquisition procedure and, if applicable, already applied processing steps. 4. Usage of the guidelines The guidelines were used in the project ‘‘Road2Simulation” for a prototypical road survey. A 14 km long test track in Stuttgart, Germany, was selected consisting of urban, non-urban, tunnel and motorway sections. The road survey was conducted in Q2 2016 and the outcome was available in Q4 2016. The resulting road network will be provided in OpenDRIVE and OpenCRG format (which is also included in the guidelines), a 3D model with the corresponding environment for driving simulator usage is generated in OpenSceneGraph format based on different cadastral data sources and using the ‘‘Virtual World” approach (Richter et al., 2016). Fig. 7 shows the pre-processed survey data compared in a GIS visualisation with the cadastral data (e.g. digital terrain model as well as digital city model and tree cadastre) of the city of Stuttgart. The surveying contractor was selected in a two-staged process where all applicants had to qualify by applying the guidelines to a sample intersection. We received ten replies for our request from different survey companies. After that six qualified contractors were called for an offer to survey the test track according to the guidelines and deliver a section of surface data, too. The bids ran one third of comparable bids requesting for direct delivery in domain-specific simulator formats.
Fig. 4. Intersection with connections for each cardinal direction.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
7
Fig. 5. Decompiled complex intersection.
Fig. 6. Punctual information (magenta) about signage, street lighting and road furniture. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Fig. 7. Surveyed and pre-processed data in combination with cadastral data.
The contractor was able to adopt the guidelines easily without crucial misunderstanding. In the beginning there was a loss of time because of a homogenisation issue of the laser scanning raw data but the main work was done within a timeframe of two months despite the huge amount of elements that had to be considered. Nevertheless some problems occurred due to sloppy work but could be fixed with little effort because no complex mathematical modelling had to be reworked.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
8
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
After that the tool chain for automated transformation into OpenDRIVE had to be improved to comply better with real-life examples. 5. Outlook The guidelines are currently updated after validation with the help of the survey result and by gathered experience. Afterwards they will be published for free again in order for other companies and authorities to use and adapt them for costefficient gathering of road data. This should result in a win-win-situation because the additionally surveyed attributes help governmental data, for example, to become interesting for third parties, especially for the automotive domain. It should also boost the open data movement by providing more machine-processable cadastral data and support the possibility to reduce the dependency on specialised data providers to survey and manage large-scale road network data. The guidelines and the simplified data format will be used for larger-scale surveying in upcoming test sites and corresponding projects that need high-precision data for development and test of automated driving applications. Additionally it is being planned to publish the tool chain for transforming the simplified data into OpenDRIVE and/or NDS, too. Later on the guidelines can be extended for additional description formats as well as for further road situations such as parking spaces or specific conditions in other countries. Also the need of describing the environment next to the road is arising. Buildings, street vegetation and the terrain should also be modelled. Thus, kind of a ‘‘general world description format” is necessary. It will be challenging to include all details and specialities in a feasible way to represent e.g. buildings realistically. 6. Conclusion The need for real-world data to test advanced assistance and automation system is growing fast. Currently the acquisition of ready-to-use real-world data for driving simulator applications is expensive. There is no adoptable approach available to minimise these efforts. Thus, a solution for reducing the complexity of surveying and transforming the geodata was developed by DLR in partnership with OEMs. Key element of this solution is an intermediate, discrete geodata format, which includes necessary metadata for transformation. The data model together with modelling suggestions and examples for typical road situations were combined into guidelines and made available free of charge. The developed guidelines are tested with a prototypical survey and data transformation and will be updated for new formats and additional road situations. Funding This work based on the experience and work of DLR that was funded by the Helmholtz Association. The development of the guidelines and simplified data model was funded by Audi, BMW, Daimler, Porsche and Volkswagen. Acknowledgements We would like to thank Hartmut Friedl and Thomas Ruppert from DLR’s Remote Sensing Data Center, Oberpfaffenhofen, Germany, for the support in dealing with the cadastral data. References Boer, E. R., Penna, M. D., Utz, H., Pedersen, L., Sierhuis, M. (2015). The role of driving simulators in developing and evaluating autonomous vehicles. In Driving Simulator Conference 2015 Europe Germany, 16–18 Sep. 2015, Tübingen, Germany. ISBN 978-3-9813099-3-5. Dupuis, M., et al., (2015). OpenDRIVEÒ Format Specification, Rev. 1.4 Issue H.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004
A. Richter, M. Scholz / Transportation Research Part F xxx (2017) xxx–xxx
9
Richter, A., Scholz, M., Friedl, H., Ruppert, T., Köster, F. (2016). Challenges and experiences in using heterogeneous, geo-referenced data for automatic creation of driving simulator environments. Simulation-transactions of the society for modeling and simulation international, 92(5), 437–446. SAGE Publications. http://dx.doi.org/10.1177/0037549716641201. ISSN 0037-5497. Richter, A., Friedl, H., Scholz, M. (2016). Beyond OSM – alternative data sources and approaches enhancing generation of road networks for traffic and driving simulations. In SUMO2016 – Traffic, Mobility, and Logistics. 23–25 May 2016, Berlin, Germany.
Please cite this article in press as: Richter, A., & Scholz, M. Deploying guidelines and a simplified data model to provide real world geodata in driving simulators and driving automation. Transportation Research Part F (2017), http://dx.doi.org/10.1016/j.trf.2017.04.004