- Email: [email protected]
A dataset for human localization and mapping with wearable sensors
Accepted Manuscript A dataset for human localization and mapping with wearable sensors Carmine Tommaso Recchiuto, Antonello Scalmato, Antonio Sgorbissa
PII: DOI: Reference:
S0921-8890(17)30036-2 http://dx.doi.org/10.1016/j.robot.2017.08.004 ROBOT 2892
To appear in:
Robotics and Autonomous Systems
Received date : 16 January 2017 Revised date : 30 June 2017 Accepted date : 5 August 2017 Please cite this article as: C.T. Recchiuto, A. Scalmato, A. Sgorbissa, A dataset for human localization and mapping with wearable sensors, Robotics and Autonomous Systems (2017), http://dx.doi.org/10.1016/j.robot.2017.08.004 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
A Dataset for Human Localization and Mapping with Wearable Sensors Carmine Tommaso Recchiuto, Antonello Scalmato, Antonio Sgorbissa Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genova, via all’Opera Pia 13,16145 Genova
Abstract The process of acquiring a complete dataset is usually a complex and time-consuming task. This is particularly true when speaking about Simultaneous Localization and Mapping (SLAM) approaches implemented with human operators wearing sensors in outdoor environments, where many problematics arise during the data acquisition. The subject is relatively recent, and there is therefore a limited number of datasets available to the scientific community. The main objective of this article is to describe and analyze a 35-minutes dataset (publicly shared) acquired during the walking process (outdoor, in the city centre of Genova) of a human operator endowed with a laser scanner and two IMUs (accelerometer, gyroscope and compass). The dataset is composed both of the raw data of the sensors and of the online estimated human odometry. Moreover, the path of the operator has been chosen in order to include multiple loops. The dataset has been made available in a format compatible with the ROS (Robots Operating Systems) environment and tested with a popular algorithm for SLAM, usually used for robotic applications, giving some preliminary insights about the basic set of wearable sensors necessary for mapping as well as on the human odometry estimation process. Keywords: SLAM, datasets, step detection, human odometry
1. Introduction The Simultaneous Localization and Mapping (SLAM) problem usually refers to the capability of a mobile robot to be placed in an unknown location in an unknown scenario and incrementally build a consistent map of the environment, while simultaPreprint submitted to Robotics and Autonomous Systems
June 30, 2017
5
neously determining its location within the map [1]. However, in some situations, a human operators or even a team of persons may have the need to map the surrounding environment, that can be partially or completely unknown, or GPS-denied. Examples of these situations are operations of Search&Rescue, where rescuers and firefighters should move in complex scenarios, possibly cluttered with obstacles and debris. In
10
such environments, buildings are likely to be completely or partially collapsed, which makes pre-existing maps not very useful: moreover, GPS signal can be partially or totally occluded. From the other side, human rescuers need to precisely know the surrounding environment, in order to be sure to have monitored all possible areas. A practical example of this situation was in the earthquake of magnitude 6.2 that
15
recently happened in Italy, at Amatrice. Almost 300 persons were killed in the earthquake, and aerial images taken by drones showed houses and walls collapsed and swathes of the city completely flattened (Figure 1).
Figure 1: Aerial view of Amatrice after the recent earthquake. Image taken from Reuters.
In this context, the implementation of a system able to localize a rescuer, endowed with wearable sensors, that walks among partially collapsed buildings while simulta20
neously building a map of the environment, can play a key role to ensure a prompt aid. However this problem has not received a great interest from the scientific commu2
nity. Indeed, only in few cases the SLAM approach involved the presence of humans and pedestrians embedded with wearable sensors: one of the few examples can be 25
found in [2], that used head-mounted inertial sensors and laser scanners in order to generate both localization information and 2D environment maps. This approach is interesting, since it does not require a priori information and user-deployed markers or beacons. The laser scanner was mounted on an helmet of the user, together with an Inertial Measurement Unit (IMU), able to estimate the head motion and correct the
30
scan data. The human odometry was estimated by implementing a simple step detection process into the motion model. However the approach has been proved to work only in indoor environments. Another interesting approach has been described in [3], where a handheld mapping system carried by a human operator has been used in order to capture cultural heritage
35
sites. In this case, the human operator is carrying a 2D laser scanner, endowed with an IMU; the system also includes a small laptop computer for recording the data and a camera rigidly mounted to the laser scanner handle. This approach generates 3D point clouds map of the environment, as the operator holding the device walks through it. The presence of datasets could be a powerful instrument for boosting the devel-
40
opment of novel techniques for Human Localization and Mapping and testing new algorithms. Indeed, the demand of the scientific community for publicly available and peer-reviewed datasets is extremely high. In past years, an editorial of the International Journal of Robotics Research (IJRR) encouraged researchers in publishing data papers, i.e. articles presenting datasets to be released to the robotics community [4].
45
One aspect that was underlined in the article is that acquiring high-calibre data is a very complex task, that can represent a high barrier, in terms of time and money, for many research groups, and generally speaking, it can slow down the development of novel algorithms. Another fundamental aspect is that releasing high-quality datasets can avoid overhead in collecting experimental data and it can facilitate a direct com-
50
parison between different works. Given the nature of the Localization and Mapping problem, almost all available datasets SLAM-related are usually acquired using mobile robots. While this aspect allows the possibility of having a wide number of deployed sensors, it can be a limiting 3
factor when dealing with human operators and wearable sensors. 55
The proposed work aims at filling this gap, presenting a novel dataset acquired in a complex outdoor scenario, the city centre of Genova, crowded and partially GPSdenied, and a technique for estimating human odometry that allows to generate online maps and to localize the human operator. Indeed, in such a scenario, as for the Search&Rescue case, the implementation of wheeled robots is particularly difficult,
60
given the reduced size of the streets and the presence of uneven ground and stairs. The dataset has also been off-line processed, demonstrating that the data acquired and the step detection procedure are sufficient for a robust map reconstruction. Moreover, the analysis of the dataset gives some insights in relation to the acquisition techniques, showing the importance of estimating both head and body frames, and the robustness
65
of the step detection process. The paper is structured as follows. Section 2 describes the most relevant SLAMrelated datasets shared by the scientific community, available in online open platforms. Section 3 describes the wearable sensors used for data acquisition and presents the structure of the proposed dataset, named the Zena1 dataset, and Section 4 shows pre-
70
liminary results obtained applying a common SLAM algorithm to the dataset. Finally Section 5 presents conclusions. 2. Related Work: SLAM Datasets Given the interests of the scientific community there is already a huge number of datasets available related to SLAM aspects. Some of the most relevant ones can be
75
found in the OpenSLAM platform [5], a community of researchers devoted to the issue of Localization and Mapping. One of the principal features of the platform is to offer the source code of a wide variety of SLAM algorithms, that can be therefore modified by the other researchers. Beyond this repositories collection, the website gives the possibility of testing novel algorithms allowing the download of a number of datasets
80
commonly used by the robotics community. 1 Zena
is the name for Genova in the local dialect.
4
Among the most relevant datasets, the Cheddar Gorge Data Set, by Simpson, Cullip and Revell [6] is intended for the multiple disciplinary research areas of ground based vehicle autonomy and field robotics. This dataset has been acquired by using a Wildcat platform and it is intended for being a basis for large scale autonomous un85
manned ground vehicles (UGVs). The Wildcat robot was endowed with stereo vision cameras and a high definition (1920 x 1080) camera, a FLIR infrared camera, a Velodyne LIDAR, a complete Inertial Navigation System (INS), with a GPS receiver, local differential GPS, OmniSTAR (a satellite-based augmentation system service provider) subscription and a IMU, and four wheel distance encoders. The duration of the ac-
90
quisition is of 1 hour ca., with an average speed of the robot of 31.5 km/h. Another valid example is the Navlab SLAMMOT Dataset [7], that provides the internal odometry of the Jeep used for the experimental phase, and data acquired with a standard and a differential GPS, gyroscopes and magnetometers, a proximity laser scanner and an omnidirectional camera. In a similar way, the KTH SLAM Datasets [8] have been
95
collected with a ATVR robot (iRobot) in the area outside the laboratory building. The robot was moving on its own, therefore some of the decisions are not the best from the map-making point of view. The datasets provide odometry (by means of the wheels encoder of the ATRV), inertial data (with kalman filtered pitch and roll estimate), SICK data and fused data-reckoning estimates, with incremental covariance.
100
Another open platform that is a fundamental resource for SLAM research is the RADISH repository [9], that provides also a collection of standard robotics data sets. However, the majority of these datasets are related to indoor environments. A valid example is the CSCMezzanine dataset, where the file contains the raw sensor data acquired by a single Pioneer2DX, equipped with internal encoders and a SICK LMS
105
200 laser range finder, during a tour at the California Science Center, with a human operator teleoperating the robot. The same setup, but in a multi-robot scenario, has been adopted for the acquisition of the ApHill dataset. Bigger datasets in the platform are the ones collected by the research group of the University of Freiburg [10]. Also in this case, a Pioneer robot has been used, endowed with a laser and the possibility of
110
estimating internal odometry. Finally, a further wide variety of datasets have been collected in the MRPT robotics 5
datasets platform [11], where the datasets are endowed with a reliable ground truth. Providing a ground truth is specially challenging in the case of visual SLAM, where the world model is 3-dimensional and the robot path is 6-dimensional. In this case, the 115
vehicle was endowed with twelve sensors: laser scanners (two Hokuyo UTW and three SICK-LMS), inertial measurement units, GPS receivers (both consumer-grade DGPS and GPS Real Time Kinematic) and color cameras. A common aspect of all the datasets here analysed is that they are all taken with robots, that give the possibility of easily calculate the odometry by using some com-
120
mon techniques (i.e., wheel encoders, IMUs, ...). Examples of datasets that have been collected from human operators walking while wearing the sensors or holding them in their hands exist: among them, the most relevant ones have been made available to the robotics community by the Team HECTOR. However it is worth saying that these datasets are taken indoor, in structured environments and they have been solved using
125
the HectorSlam algorithm, that does not require the human odometry as an input [12]. The datasets, available online at the Hector-Team googlecode repository [13], contain therefore data related to an Hokuyo UTM-30LX LIDAR systems and a low cost IMU. The dataset collected by the team of the University of Oxford [14] is also worth of mention. They focused on autonomous navigation in human working environments,
130
with a relative topometric approach to the SLAM problem. To this aim they collected a huge dataset of 850,000 images, covering 142 km of Southern England. Images, captured at 20Hz, are taken both with robots and with human operators, using various forms of transport, including foot, bycicle, train, subway, escalator, rickshaw, puntingboat and ferris wheel. As they notice, human-like movement in urban spaces is very
135
different, and substantially more challenging than processing stable robot data. Finally, the datasets collected by the CSIRO Computational Informatics (CCI) group of Brisbane, have been also collected by human operators as demonstrated in [3], and they are available online [15]. The handheld mapping system has been already described in the Introduction. In this case the datasets are acquired in large outdoor environment and
140
without the usage of robots; however, since the mapping system is based on 3D point cloud maps consisting of hundred million of points, the data cannot be processed in real-time during acquisition. 6
The current proposed work aims at offering a novel dataset acquired with sensors directly worn by a human operator, by means of a helmet and of a sensoryzed belt. 145
The novelty with respect of previous datasets can be mainly found in the utilization of a system of wearable sensors placed taking into account the kinematic constraints of the human SLAM problem, allowing a real-time estimation of the human odometry and therefore an online processing of the acquired data. Moreover, the acquisition was performed outdoor, in a wide and unstructured scenario (a part of the historical center
150
of Genova) 3. The Zena Dataset 3.1. Wearable sensors The main idea of the approach is to acquire the dataset with a set of sensors that can be worn by a firefighter in a Search&Rescue scenario. Therefore, the setup for
155
the acquisition is based on the following sensors: a laser scanner hokuyo UTM-30LX, with 30m and 270 degrees of scanning range and two IMUs PhidgetSpatial Precision 3/3/3, that integrate a 3-axis compass, a 3-axis gyroscope, and a 3-axis accelerometer. The laser scanner has been mounted on the helmet in order to avoid possible occlusions (that could have been occurred having it on the torso or on the waist). One of the two
160
IMUs is also placed on the helmet, acquiring information about the orientation and the movements of the head (in the same frame of the laser scanner), while the other IMU has been fixed to a belt, and placed on the waist of the human operator. In this way, it is possible to acquire distinct information related to the frames head and body; as it will be shown in the following, this aspect is a fundamental peculiarity of the
165
proposed dataset. Figure 2 shows the human operator during the acquisition of the dataset. The data of the sensors are acquired and stored by means of a notebook placed in the backpack (Asus Zenbook UX501VW). 3.2. Data format The Zena dataset refers to a portion of the Genova city centre, in an area approxi-
170
mately of 200 x 130 meters, while the path travelled by the human operator can be estimated in 1.450 meters (without considering the repeated passages on the same roads). 7
Laser Scanner IMU (helmet)
Backpack with the PC
Smartphone for the online visualization of the map IMU (waist) Figure 2: Human operator and wearable sensors.
During the path, fourteen loops have been closed, as it can be seen in Figure 3. In the following, with the term loop, we always mean inner loops, i.e., we ignore bigger loops that can include smaller loops inside them. 175
An overall description of the proposed dataset is given in Table 1. The dataset is shared both as comma-separated values (csv) files and as ROS file, exploiting the architecture based on ROS topics [17]. Indeed, ROS has become a standard in robotics research and facilitates integration of contributions by various researchers. Moreover, a great amount of the most common algorithms for SLAM
180
purposes are already shared as ROS applications. Thus, the dataset has been also saved as a bag file, a format for storing ROS message data, widely used from researchers to record and store datasets. Going more into details, the structure of the dataset is the following: • Laser Data: direct acquisition of the laser scanner, at 40 Hz. The laser scanner
185
returns 720 laser range data every 25 milliseconds, with and angular resolution of 0.375 degrees (from -135 to +135 degrees), and a maximum range of 30m. 8
Figure 3: Area where the dataset has been acquired. The red lines indicate the path followed by the human operator. Image taken with Google Earth.
The ROS message is encoded as a laser scan message, while each line of the csv file is structured with a timestamp and the 720 ranges of the laser (Table 1, row I). 190
• IMU (head) data: raw data of the IMU board placed on the helmet at 125 Hz. Data related to the acceleration, the angular velocity and the heading on the
three axes are saved every 8 milliseconds. The ROS data are encoded with a custom ROS message, which has the same structure of the csv file: timestamp, acceleration (x, y, z), angular velocity (x, y, z), orientation (x, y, z) (Table 1, row 195
II). • IMU (waist) data: raw data of the IMU board placed on the waist at 125 Hz. The
structure of the ROS message and of the csv file is the same as before (Table 1, row III).
While these three information have been directly acquired with the described set of 9
Laser data
Frequency (Hz)
Number of Data
CSV File
40
83945
laserscan.csv
CSV Data Format Timestamp (ns), Scan Ranges [0 to 719] (m).
ROS topic
Ros message type
/scan
sensor msgs/laser scan
/imudata
custom msgs/imu
/imudata
custom msgs/imu
/base scan
sensor msgs/laser scan
/odompose
geometry msgs/Twist
/headpose
geometry msgs/Twist
Timestamp (ns), IMU (head)
125
261487
Acceleration [x, y, z] (m2 /s),
imu head.csv
Orientation [x, y, z] (rad), Angular velocities [x, y, z] (m/s). Timestamp (ns),
IMU (body)
125
261497
Acceleration [x, y, z] (m2 /s),
imu body.csv
Orientation [x, y, z] (rad), Angular velocities [x, y, z] (m/s).
Laser data (synchronized with the
Timestamp (ns),
40
83904
laserscan.csv
125
254555
odom body.csv
Scan Ranges [0 to 719] (m).
Odometry) Odometry (head) Odometry (body)
Timestamp (ns), Position [x, y] (m), Heading [θ ] (rad). Timestamp (ns), 125
254565
odom head.csv
Position [x, y] (m), Heading [θ ] (rad).
Table 1: Data format of the presented dataset
200
wearable sensors, the Zena dataset has been also integrated with the human odometry, online estimated by means of the IMU data. Indeed, the almost totality of SLAM algorithms requires an estimation of the odometry (of the robot or, in this case, of the human operator), and therefore the information coming from the IMU boards should be processed in order to obtain an estimation of the position of the user. Additionally, this
205
processing should be done online, giving the Search&Rescue context and the necessity of generating a map while the human operator is walking. In the current approach, the estimation of the human odometry has been performed using a step detection procedure. This procedure is described in [16] and it is based on an algorithm for peaks (steps) discovering. The acceleration measured by the IMU
210
is analysed and, assuming that there is a periodic pattern in the signal, the steps are detected. Briefly, the accelorometer raw signal is processed with a low-pass filter, in order to remove high frequencies components, and after that, the vertical acceleration data are fed to a standard signal analysis algorithm to identify the peaks2 . Human ori2 The
peak
identification
procedure
is
available
10
at:
http://www.mathworks.com/
entation and step directions are determined using the compass integrated in the IMU 215
boards (filtered). The step length can depend on different factors among which the user’s height, but scan matching algorithms used in common SLAM algorithms well compensate the movement and estimate the user’s position at each step (same considerations for the orientation). In our model, the step has been fixed to 0.6 meter, but tests have been performed varying this parameter (results will be shown in Section 4).
220
For the current purposes, this approach has been implemented online, using temporal windows, in which the signal is analysed and the steps are detected. The estimation of the optimal window length is not trivial. At this aim, different window lengths have been tested: it has been noticed that with a window length greater than one second, steps were always detected with a good approximation. Indeed, an offline analysis of
225
the complete raw signal of the accelerometer detects a comparable number of steps. The algorithm estimates the user’s motion analyzing the acceleration pattern as shown in Figure 4. When a step is detected, the motion of the user is estimated with a step forward, the motion is spread between the previous spike and the detected one and the information is integrated with the heading (measured with the compass) in order
230
to determine the step direction. It can be seen how the greater is the chosen temporal window, the higher is the difference between the real motion of the operator and the estimated one: indeed, the step is interpolated from the start of the window (or from the previous spike) to the end of the step, and in the case of a single step in a wide temporal window, substantial errors can arise. For this reason, a temporal window length of one
235
second (the minimum possible length that allows a precise detection of the step) has been chosen. Finally, odometry data should be synchronized with the laser ranges information, i.e. laser data should be delayed of a temporal window in order to be used together with the online estimated odometry. Therefore, the dataset has been enriched with the following data:
240
• Human Odometry (head): estimation of the odometry using the accelerometer and the compass mounted on the head. The x and y position and the orientation
matlabcentral/fileexchange/25500-peakfinder
11
吀攀洀瀀漀爀 愀氀 眀椀 渀搀漀眀
吀攀洀瀀漀爀 愀氀 眀椀 渀搀漀眀
Figure 4: Accelerometer data (up) and estimation of the human odometry. For the sake of clearness, in this case the motion has been estimated all along the x axis. The red lines correspond to the end of the step detection.
θ along the z axis are estimated each 8 ms (with the spreading process shown before). The ROS message is encoded as a twist message, while each line of the csv file is structured with a timestamp, the x and y position, and the heading θ 245
(Table 1, row V). • Human Odometry (body): estimation of the odometry using the accelerometer and the compass mounted on the waist. The spreading process, and the structure of the ROS message and of the csv file is the same as before (Table 1, row VI). • Laser scan (synchronized with the odometry): laser scanner data delayed of one
250
second (the odometry temporal window). The information is already contained in the Laser Data csv file; however, for helping the final users, an additional ROS topic has been added to the bag file (Table 1, row IV).
12
The whole dataset, including all csv files and the ROS bag file, is made available at: https://github.com/laboratoriumunige/zenadataset. 255
It is worth saying that the process of estimating human odometry is inevitably affected by a certain error: this is due to the fixed step length, to errors in the step detection phases, and also to the fact that the compass measurement is often affected by buildings’ structures, such as steel frame, iron pipes. However, as it will shown in the following paragraph, estimating the position of the user with the proposed approach
260
allow the SLAM algorithm to reconstruct a reliable map. 4. Analysis This final section aims at giving a preliminary validation of the proposed dataset, using it as input of a commonly used algorithm for robotics SLAM, in order to give some starting references to the users and to investigate some aspects of the dataset itself
265
and of the chosen set of wearable sensors. The Zena dataset has been investigated by using the KartoSLAM algorithm [18]. It must be here reminded that the utilization of the KartoSLAM algorithm is possible only if there is an estimation of the odometry of the robot (or in this case of the human operator), since it requires the pose and orientation of the body and laser frames with
270
respect to the world frame. Thus, the odometry estimated as shown in the previous section has been used as input. The KartoSLAM package is a robust algorithm for Localization and Mapping, based on the graph-SLAM technique. In the graph-SLAM approach the measurements are represented as a graph, in which every node represents a local map, and it is la-
275
beled with the location at which the measurement was taken, and the edges between two nodes encode the spatial information arising from the alignment of the connected measurements (they can be seen as the spatial constraints between two nodes). The problem is then twofold: identify the constraints based on sensor data, and correct the poses of the robot to obtain a consistent map of the environment. There are differ-
280
ent strategies aimed at solving these problems: among the most relevant, the Sparse Pose Adjustment (SPA) and the G2 O (General Graph Optimization) are adopted in
13
the KartoSLAM approach. These methods follow the concept of bundle-adjustment (BA) algorithms originally developed for computer vision, exploiting the sparsity of the measurement graph so as to speed up computation. 285
In the proposed work, the SPA approach has proved to be the more reliable one, in relation to the chosen sensors worn by the human operator, and to the structure of the environment, and it has therefore been used during the analysis of the dataset. A preliminary work has been performed for the calibration of the parameters, in order to find a configuration that allowed to profitably complete the mapping task; in particular
290
the following parameters have been adapted for the particular outdoor context: • range threshold: it sets the scanning range considered by the algorithm for scan matching. It has been set to 8 meters.
• minimum travel distance: it is the minimum travel distance between scans, i.e.
the mapper will use the data from the new scan only if a new scan’s position is
295
more than this value from the previous scan. It has been set to 0.5 m (default value was 0.2 m) • minimum travel heading: this parameter sets the minimum heading change, i.e.
the mapper will use the data from the new scan only if a new scan’s heading is more than this value from the previous scan. It has been set to 0.06 rad (default
300
value was 0.549 rad) • correlation search space dimension: the size of the search grid used by the matcher when matching sequential scans. It has been set to 1.0 m (default value is 0.3 m). The KartoSLAM algorithm has been applied to the dataset directly during the ac-
305
quisition of the data, in order to test the capacity of the system of generating a map online, a crucial aspect in a Search&Rescue scenario. However, because of the GraphSLAM structure, the number of nodes and the size of the graph increase with the user’s motion, and after some minutes the map can not be easily handled in real-time (i.e, with no visible delays between sensor acquisition and mapping). Moreover, the size of
14
310
the map that can be handled strictly depends on the onboard processing power. Processing the current dataset online with the notebook used for the acquisition (i.e., Asus Zenbook UX501VW with a Intel Core i7-6700HQ) the KartoSLAM algorithm can generate a valid map for around 10 minutes, closing five loops. After this point the number of nodes of the graph describing the total map becomes too high (around 2000
315
nodes) and the corresponding matrices can not be correctly handled in real-time. The same dataset has been tested with a different CPU, a Intel Core i7-6700K, and in this case the algorithm is able to generate a valid and reliable online map for 990 seconds, closing seven loops and generating 2934 nodes. Figure 5 shows the portion of the map than can be generated online with the described hardware.
Figure 5: Biggest map obtained processing the data online. Seven loops are closed, and 2934 nodes are generated; after 990 seconds, the size of the graph is too big to be processed online, and errors arise in the mapping process.
320
During the acquisition, the user can monitor the map building process by using a custom-built android application, that exploiting the ROS-for-android version, connects to the map topic (generated by the kartoSLAM algorithm running on the notebook) and may display the map on a common smartphone. A screenshot of the application is shown in Figure 6.
15
Figure 6: Screenshot of the android application for online map visualization
325
In order to process the whole dataset, and thus obtain some insights about the role of each sensor (belt IMU, head-mounted IMU, head-mounted laser scanner) for producing accurate maps and on the precision of the human odometry estimation, the dataset has been also processed offline with the KartoSLAM algorithm. First of all, the dataset has been tested in relation to the set of wearable sensors
330
used for the acquisition: in particular, the advantages arising from using two IMUs (on the head and on the waist) have been evaluated. Indeed, as mentioned before, the KartoSLAM algorithm requires two different mobile frames: • Frame body, that describes the human body’s position and orientation on a plane. • Frame laser, that describes the position and orientation of the laser on a plane.
335
The utilization of these frames implies the ignoring of 3D position and attitude, thus approximating the system as if it were a wheeled robot moving on a flat ground. This is a strong assumption that, however, proves to be acceptable during experiments performed in a urban settings, where we are more interested in detecting free spaces 16
and occupied spaces determined by buildings, than in drawing an accurate 3D repre340
sentation. Tests have been performed with these frames in different configurations as described in the following: Configuration a: • x, y and θ of body frame are measured through the human odometry approach
345
described in Section 3, using the IMU located in the belt (i.e. x and y and θ from
the Human Odometry (body) of the dataset); • x, y of laser frame are the same of body frame (plus a constant offset); • θ of laser frame is measured using the compass located on the helmet (the heading θ from the Human Odometry (head) of the dataset).
350
Configuration b: • x, y and θ of body frame and x, y of laser frame are the same of Configuration a; • θ of laser frame is assumed to be the same measured by the compass located on the waist, i.e., θ of laser frame = θ of body frame.
In other words, this configuration corresponds to the frames description that could be 355
done using a single IMU located on the waist (i.e., only data from the Human Odometry (body) of the dataset). Configuration c: • x, y and θ of laser frame are calculated using the IMU located in the helmet (i.e., x and y and θ from the Human Odometry (head) of the dataset);
360
• x, y of body frame are the same of laser frame (plus a constant offset); • θ of body frame is assumed to be the same measured by the compass located on the helmet (the heading θ from the Human Odometry (head) of the dataset).
In other words, this configuration corresponds to the frames description that could be done using a single IMU located on the helmet (i.e., only data from the Human 365
Odometry (head) of the dataset). 17
The percentage of loops closed, averaged over five repeated runs, using these different frames configuration has been considered as performance indicator. The repetition of the runs is necessary since the KartoSLAM algorithm has stochastic components related to the handling of the odometry and to the scan matching algorithm; therefore, 370
the maps obtained, with the same inputs and parameters, can slightly differ from each other. As expected, the Configuration a gives the best results, allowing, at each run, to close almost the totality of the loops and obtaining a reliable map, since it takes into account the orientation of the head with respect of the body. On the other side, Config-
375
uration b allows the algorithm to close on average only the 39.29% of loops, making a wrong correlation between the laser scan data (in the head frame) with the body frame, losing precision in the scan matching process. Finally, Configuration c allows, on average, for the closure of the 57.14% of loops, neglecting the kinematic chain of the human body, with errors in the calculation of its position and orientation. Indeed,
380
even if humans are potentially able to move in all directions on the xy plane (i.e., not depending on the heading of their body and of their head), nonetheless the preferred direction of motion is usually forward. Then, treating them as a unicycle robot with a panning laser appears to be more convenient than modelling them as a holonomous robot.
385
Figure 7 shows two examples of the map calculated by using frames in Configuration a and in Configuration c.
18
1
2 1
3 4 5
6
8
7
9
11
2
10
3
12
4
5
13
14
6
Figure 7: On the left, map obtained using Configuration a. All 14 loops are closed. On the right, map obtained using Configuration c. Only 6 loops are closed, and the map building process is not reliable.
The second aspect analized is strictly related to the process of the step detection. Indeed, it has been described how the step detection process assumes that the step length is equal to 0.6 meter. This has been estimated in relation to the average walking 390
pattern of the user, but obviously the step detection and the mapping processes should be robust in relation to the variation of this parameter. To this aim, the dataset was tested (with frames in Configuration a) while varying the value of the step, from 0.4 to 1.0 meter. Also in this case, the values are averaged over five runs. It can be seen how the variation of the step size has some influences only if the step size goes below
395
a certain value (the percentage of loop closures is below 75% if the step size is 0.4m); otherwise, the percentage of loop closures is always higher than 90%, showing that the step detection process is robust to the variation of the step size. Table 2 summarizes the results obtained in the analysis of the dataset.
19
Frame Configuration Step size(m) Percentage of loop closures
a
b
c
a
a
a
a
a
a
0.6
0.6
0.6
0.4
0.5
0.7
0.8
0.9
1.0
97.14 %
39.29%
57.14%
71.40%
91.43%
97.14%
95.70%
94.28%
92.86%
Table 2: Percentage of loop closures of the maps, with different frames configurations and different step sizes.
5. Conclusions 400
While the possibility for first responders and human operators of mapping an unknown scenarios while walking can be of the utmost importance in Search&Rescue scenarios, this research subject has not been so much taken under investigation from the research community. As a consequence, there are few available datasets related to this topic, and they are all limited to indoor environments.
405
The presented article describes the acquisition procedure of a 35-minutes dataset for human SLAM in a complex outdoor scenario, such as the historical centre of Genova. A laser scanner and two IMUs have been used as wearable sensors. For the estimation of the human odometry, a procedure for the online detection of the step, using the two IMU boards, has been implemented.
410
The analysis of the Zena dataset, using a popular algorithm for SLAM specifically tailored for mobile robots, shows how the proposed procedure allows reconstructing the map of the surrounding environment, in particular when using all data of the wearable sensors, i.e., two different frames for the laser position (mounted on the helmet of the user) and for the body position. Finally, the step detection procedure has been proved
415
robust to variations of the step size. Future works will be focused in integrating strategies for long-term maintenance of the graph [19] [20], to avoid the real-time issues arising when processing the dataset online.
20
6. References 420
[1] Durrant-Whyte, H., & Bailey, T. (2006). Simultaneous localization and mapping: part I. IEEE robotics & automation magazine, 13(2), 99-110. [2] Cinaz, B., & Kenn, H. (2008, September). HeadSLAM-simultaneous localization and mapping with head-mounted inertial and laser range sensors. In 2008 12th IEEE International Symposium on Wearable Computers (pp. 3-10). IEEE.
425
[3] Zlot, R., Bosse, M., Greenop, K., Jarzab, Z., Juckes, E., & Roberts, J. (2014). Efficiently capturing large, complex cultural heritage sites with a handheld mobile 3D laser mapping system. Journal of Cultural Heritage, 15(6), 670-678. [4] Newman, P., & Corke, P. (2009). Editorial Data Papers-Peer Reviewed Publication of High Quality Data Sets. The International Journal of Robotics Research,
430
28(5), 587-587. [5] Stachniss, C., Frese, U., & Grisetti, G. (2007). Openslam. org. [6] Simpson, R., Cullip, J., & Revell, J. (2011). The cheddar gorge data set. Tech. Rep. [7] Wang, C. C., Duggins, D., Gowdy, J., Kozar, J., MacLachlan, R., Mertz, C., ...
435
& Thorpe, C. (2004). Navlab slammot datasets. Pittsburgh, PA: Carnegie Mellon University. [8] http://www.nada.kth.se/∼johnf/kthdata/dataset.html. Online, January 2017 [9] Howard, A., & Roy, N. (2003). The robotics data set repository (radish). [10] Stachniss, C. (2006). Exploration and mapping with mobile robots (Doctoral dis-
440
sertation, University of Freiburg). [11] Blanco, J. L., Moreno, F. A., & Gonzalez, J. (2009). A collection of outdoor robotic datasets with centimeter-accuracy ground truth. Autonomous Robots, 27(4), 327-351.
21
[12] Kohlbrecher, S., Von Stryk, O., Meyer, J., & Klingauf, U. (2011, November). A 445
flexible and scalable slam system with full 3d motion estimation. In 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics (pp. 155-160). IEEE. [13] https://code.google.com/archive/p/tu-darmstadt-ros-pkg/. Online, January 2017 [14] Sibley, G., Mei, C., Reid, I., & Newman, P. (2010). Vast-scale outdoor navigation
450
using adaptive relative bundle adjustment. The International Journal of Robotics Research, 29(8), 958-980. [15] https://data.csiro.au/dap/. Online, January 2017 [16] Bruno, B., Mastrogiovanni, F. & Sgorbissa, A. (2015). HOOD: a Real Environment Human Odometry Dataset for Wearable Sensor Placement Analysis. In In-
455
telligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, At Hamburg, Germany. [17] Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., ... & Ng, A. Y. (2009, May). ROS: an open-source Robot Operating System. In ICRA workshop on open source software (Vol. 3, No. 3.2, p. 5).
460
[18] Santos, J. M., Portugal, D., & Rocha, R. P. (2013, October). An evaluation of 2D SLAM techniques available in robot operating system. In 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) (pp. 1-6). IEEE. [19] Walcott-Bryant, A., Kaess, M., Johannsson, H., & Leonard, J. J. (2012, October). Dynamic pose graph SLAM: Long-term mapping in low dynamic environments.
465
In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1871-1878). IEEE. [20] Einhorn, E., & Gross, H. M. (2015). Generic NDT mapping in dynamic environments and its application for lifelong SLAM. Robotics and Autonomous Systems, 69, 28-39.
22
Carmine Tommaso Recchiuto received his Master Degree in Electronic Engineering from the University of Pisa in 2008. From 2009 to 2012 he was Research Fellow at the Scuola Superiore Sant’Anna, where he was involved in projects related to the realization of tactile sensors and humanoid robotics. From 2013 he was Research Fellow in the Department of Informatics, Bioengineering, Robotics, and System Sciences (DIBRIS), at the University of Genova, where he was involved in the PRISMA project, focussing his work in the implementation of UAVs in emergency scenarios. He received his Ph.D in 2015 in Microsystems from the University of Rome Tor Vergata. He is currently Post-Doc Fellow with the University of Genova. His main interests are in the fields of tacitle sensors, UAVs, humanoids and human-robot interaction.
Antonello Scalmato received the M.S. degree in computer science engineering and the Ph.D. degree from the University of Genoa, Genoa, Italy, in 2008 and 2011, respectively. He is a Research Fellow with the Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa. His research interests include ontology applications for mobile robotics and ambient intelligence. His current research interests include context-aware systems for smart homes and robots in intelligent environments.
Antonio Sgorbissa is Associate Professor at University of Genova, Department of Informatics, Bioengineering, Robotics, and Systems Sciences (DIBRIS), Italy. In 2000 he received his Ph.D. in Robotics from the University of Genova, and from 2001 to 2004 he was a Post Doc at Georgia Tech, University of Parma and later at University of Genova. He currently teaches Ambient Intelligence, Cooperative Robotics, and Real-Time Operating Systems at the Faculty of Engineering as well as in the European Master on Advanced Robotics (EMARO), and Computer Science at the Faculty of Psychology. His main research interests are in the area of multi-robot systems, wearable robotics, and ambient intelligence. His research interests include also planning, knowledge representation, and machine consciousness. He is author/coauthor of more than 100 international scientific papers, and of two international patents in Robotics
Highlights - A novel dataset acquired by human operators endowed with wearable sensors -
Wereable sensors are placed taking into account the kinematic constraints of the human SLAM problem
-
Human odometry is based on a step estimation process
-
The analysis of the dataset shows the importance of estimating both head and body frames, and the robustness of the step detection process.
PII: DOI: Reference:
S0921-8890(17)30036-2 http://dx.doi.org/10.1016/j.robot.2017.08.004 ROBOT 2892
To appear in:
Robotics and Autonomous Systems
Received date : 16 January 2017 Revised date : 30 June 2017 Accepted date : 5 August 2017 Please cite this article as: C.T. Recchiuto, A. Scalmato, A. Sgorbissa, A dataset for human localization and mapping with wearable sensors, Robotics and Autonomous Systems (2017), http://dx.doi.org/10.1016/j.robot.2017.08.004 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
A Dataset for Human Localization and Mapping with Wearable Sensors Carmine Tommaso Recchiuto, Antonello Scalmato, Antonio Sgorbissa Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genova, via all’Opera Pia 13,16145 Genova
Abstract The process of acquiring a complete dataset is usually a complex and time-consuming task. This is particularly true when speaking about Simultaneous Localization and Mapping (SLAM) approaches implemented with human operators wearing sensors in outdoor environments, where many problematics arise during the data acquisition. The subject is relatively recent, and there is therefore a limited number of datasets available to the scientific community. The main objective of this article is to describe and analyze a 35-minutes dataset (publicly shared) acquired during the walking process (outdoor, in the city centre of Genova) of a human operator endowed with a laser scanner and two IMUs (accelerometer, gyroscope and compass). The dataset is composed both of the raw data of the sensors and of the online estimated human odometry. Moreover, the path of the operator has been chosen in order to include multiple loops. The dataset has been made available in a format compatible with the ROS (Robots Operating Systems) environment and tested with a popular algorithm for SLAM, usually used for robotic applications, giving some preliminary insights about the basic set of wearable sensors necessary for mapping as well as on the human odometry estimation process. Keywords: SLAM, datasets, step detection, human odometry
1. Introduction The Simultaneous Localization and Mapping (SLAM) problem usually refers to the capability of a mobile robot to be placed in an unknown location in an unknown scenario and incrementally build a consistent map of the environment, while simultaPreprint submitted to Robotics and Autonomous Systems
June 30, 2017
5
neously determining its location within the map [1]. However, in some situations, a human operators or even a team of persons may have the need to map the surrounding environment, that can be partially or completely unknown, or GPS-denied. Examples of these situations are operations of Search&Rescue, where rescuers and firefighters should move in complex scenarios, possibly cluttered with obstacles and debris. In
10
such environments, buildings are likely to be completely or partially collapsed, which makes pre-existing maps not very useful: moreover, GPS signal can be partially or totally occluded. From the other side, human rescuers need to precisely know the surrounding environment, in order to be sure to have monitored all possible areas. A practical example of this situation was in the earthquake of magnitude 6.2 that
15
recently happened in Italy, at Amatrice. Almost 300 persons were killed in the earthquake, and aerial images taken by drones showed houses and walls collapsed and swathes of the city completely flattened (Figure 1).
Figure 1: Aerial view of Amatrice after the recent earthquake. Image taken from Reuters.
In this context, the implementation of a system able to localize a rescuer, endowed with wearable sensors, that walks among partially collapsed buildings while simulta20
neously building a map of the environment, can play a key role to ensure a prompt aid. However this problem has not received a great interest from the scientific commu2
nity. Indeed, only in few cases the SLAM approach involved the presence of humans and pedestrians embedded with wearable sensors: one of the few examples can be 25
found in [2], that used head-mounted inertial sensors and laser scanners in order to generate both localization information and 2D environment maps. This approach is interesting, since it does not require a priori information and user-deployed markers or beacons. The laser scanner was mounted on an helmet of the user, together with an Inertial Measurement Unit (IMU), able to estimate the head motion and correct the
30
scan data. The human odometry was estimated by implementing a simple step detection process into the motion model. However the approach has been proved to work only in indoor environments. Another interesting approach has been described in [3], where a handheld mapping system carried by a human operator has been used in order to capture cultural heritage
35
sites. In this case, the human operator is carrying a 2D laser scanner, endowed with an IMU; the system also includes a small laptop computer for recording the data and a camera rigidly mounted to the laser scanner handle. This approach generates 3D point clouds map of the environment, as the operator holding the device walks through it. The presence of datasets could be a powerful instrument for boosting the devel-
40
opment of novel techniques for Human Localization and Mapping and testing new algorithms. Indeed, the demand of the scientific community for publicly available and peer-reviewed datasets is extremely high. In past years, an editorial of the International Journal of Robotics Research (IJRR) encouraged researchers in publishing data papers, i.e. articles presenting datasets to be released to the robotics community [4].
45
One aspect that was underlined in the article is that acquiring high-calibre data is a very complex task, that can represent a high barrier, in terms of time and money, for many research groups, and generally speaking, it can slow down the development of novel algorithms. Another fundamental aspect is that releasing high-quality datasets can avoid overhead in collecting experimental data and it can facilitate a direct com-
50
parison between different works. Given the nature of the Localization and Mapping problem, almost all available datasets SLAM-related are usually acquired using mobile robots. While this aspect allows the possibility of having a wide number of deployed sensors, it can be a limiting 3
factor when dealing with human operators and wearable sensors. 55
The proposed work aims at filling this gap, presenting a novel dataset acquired in a complex outdoor scenario, the city centre of Genova, crowded and partially GPSdenied, and a technique for estimating human odometry that allows to generate online maps and to localize the human operator. Indeed, in such a scenario, as for the Search&Rescue case, the implementation of wheeled robots is particularly difficult,
60
given the reduced size of the streets and the presence of uneven ground and stairs. The dataset has also been off-line processed, demonstrating that the data acquired and the step detection procedure are sufficient for a robust map reconstruction. Moreover, the analysis of the dataset gives some insights in relation to the acquisition techniques, showing the importance of estimating both head and body frames, and the robustness
65
of the step detection process. The paper is structured as follows. Section 2 describes the most relevant SLAMrelated datasets shared by the scientific community, available in online open platforms. Section 3 describes the wearable sensors used for data acquisition and presents the structure of the proposed dataset, named the Zena1 dataset, and Section 4 shows pre-
70
liminary results obtained applying a common SLAM algorithm to the dataset. Finally Section 5 presents conclusions. 2. Related Work: SLAM Datasets Given the interests of the scientific community there is already a huge number of datasets available related to SLAM aspects. Some of the most relevant ones can be
75
found in the OpenSLAM platform [5], a community of researchers devoted to the issue of Localization and Mapping. One of the principal features of the platform is to offer the source code of a wide variety of SLAM algorithms, that can be therefore modified by the other researchers. Beyond this repositories collection, the website gives the possibility of testing novel algorithms allowing the download of a number of datasets
80
commonly used by the robotics community. 1 Zena
is the name for Genova in the local dialect.
4
Among the most relevant datasets, the Cheddar Gorge Data Set, by Simpson, Cullip and Revell [6] is intended for the multiple disciplinary research areas of ground based vehicle autonomy and field robotics. This dataset has been acquired by using a Wildcat platform and it is intended for being a basis for large scale autonomous un85
manned ground vehicles (UGVs). The Wildcat robot was endowed with stereo vision cameras and a high definition (1920 x 1080) camera, a FLIR infrared camera, a Velodyne LIDAR, a complete Inertial Navigation System (INS), with a GPS receiver, local differential GPS, OmniSTAR (a satellite-based augmentation system service provider) subscription and a IMU, and four wheel distance encoders. The duration of the ac-
90
quisition is of 1 hour ca., with an average speed of the robot of 31.5 km/h. Another valid example is the Navlab SLAMMOT Dataset [7], that provides the internal odometry of the Jeep used for the experimental phase, and data acquired with a standard and a differential GPS, gyroscopes and magnetometers, a proximity laser scanner and an omnidirectional camera. In a similar way, the KTH SLAM Datasets [8] have been
95
collected with a ATVR robot (iRobot) in the area outside the laboratory building. The robot was moving on its own, therefore some of the decisions are not the best from the map-making point of view. The datasets provide odometry (by means of the wheels encoder of the ATRV), inertial data (with kalman filtered pitch and roll estimate), SICK data and fused data-reckoning estimates, with incremental covariance.
100
Another open platform that is a fundamental resource for SLAM research is the RADISH repository [9], that provides also a collection of standard robotics data sets. However, the majority of these datasets are related to indoor environments. A valid example is the CSCMezzanine dataset, where the file contains the raw sensor data acquired by a single Pioneer2DX, equipped with internal encoders and a SICK LMS
105
200 laser range finder, during a tour at the California Science Center, with a human operator teleoperating the robot. The same setup, but in a multi-robot scenario, has been adopted for the acquisition of the ApHill dataset. Bigger datasets in the platform are the ones collected by the research group of the University of Freiburg [10]. Also in this case, a Pioneer robot has been used, endowed with a laser and the possibility of
110
estimating internal odometry. Finally, a further wide variety of datasets have been collected in the MRPT robotics 5
datasets platform [11], where the datasets are endowed with a reliable ground truth. Providing a ground truth is specially challenging in the case of visual SLAM, where the world model is 3-dimensional and the robot path is 6-dimensional. In this case, the 115
vehicle was endowed with twelve sensors: laser scanners (two Hokuyo UTW and three SICK-LMS), inertial measurement units, GPS receivers (both consumer-grade DGPS and GPS Real Time Kinematic) and color cameras. A common aspect of all the datasets here analysed is that they are all taken with robots, that give the possibility of easily calculate the odometry by using some com-
120
mon techniques (i.e., wheel encoders, IMUs, ...). Examples of datasets that have been collected from human operators walking while wearing the sensors or holding them in their hands exist: among them, the most relevant ones have been made available to the robotics community by the Team HECTOR. However it is worth saying that these datasets are taken indoor, in structured environments and they have been solved using
125
the HectorSlam algorithm, that does not require the human odometry as an input [12]. The datasets, available online at the Hector-Team googlecode repository [13], contain therefore data related to an Hokuyo UTM-30LX LIDAR systems and a low cost IMU. The dataset collected by the team of the University of Oxford [14] is also worth of mention. They focused on autonomous navigation in human working environments,
130
with a relative topometric approach to the SLAM problem. To this aim they collected a huge dataset of 850,000 images, covering 142 km of Southern England. Images, captured at 20Hz, are taken both with robots and with human operators, using various forms of transport, including foot, bycicle, train, subway, escalator, rickshaw, puntingboat and ferris wheel. As they notice, human-like movement in urban spaces is very
135
different, and substantially more challenging than processing stable robot data. Finally, the datasets collected by the CSIRO Computational Informatics (CCI) group of Brisbane, have been also collected by human operators as demonstrated in [3], and they are available online [15]. The handheld mapping system has been already described in the Introduction. In this case the datasets are acquired in large outdoor environment and
140
without the usage of robots; however, since the mapping system is based on 3D point cloud maps consisting of hundred million of points, the data cannot be processed in real-time during acquisition. 6
The current proposed work aims at offering a novel dataset acquired with sensors directly worn by a human operator, by means of a helmet and of a sensoryzed belt. 145
The novelty with respect of previous datasets can be mainly found in the utilization of a system of wearable sensors placed taking into account the kinematic constraints of the human SLAM problem, allowing a real-time estimation of the human odometry and therefore an online processing of the acquired data. Moreover, the acquisition was performed outdoor, in a wide and unstructured scenario (a part of the historical center
150
of Genova) 3. The Zena Dataset 3.1. Wearable sensors The main idea of the approach is to acquire the dataset with a set of sensors that can be worn by a firefighter in a Search&Rescue scenario. Therefore, the setup for
155
the acquisition is based on the following sensors: a laser scanner hokuyo UTM-30LX, with 30m and 270 degrees of scanning range and two IMUs PhidgetSpatial Precision 3/3/3, that integrate a 3-axis compass, a 3-axis gyroscope, and a 3-axis accelerometer. The laser scanner has been mounted on the helmet in order to avoid possible occlusions (that could have been occurred having it on the torso or on the waist). One of the two
160
IMUs is also placed on the helmet, acquiring information about the orientation and the movements of the head (in the same frame of the laser scanner), while the other IMU has been fixed to a belt, and placed on the waist of the human operator. In this way, it is possible to acquire distinct information related to the frames head and body; as it will be shown in the following, this aspect is a fundamental peculiarity of the
165
proposed dataset. Figure 2 shows the human operator during the acquisition of the dataset. The data of the sensors are acquired and stored by means of a notebook placed in the backpack (Asus Zenbook UX501VW). 3.2. Data format The Zena dataset refers to a portion of the Genova city centre, in an area approxi-
170
mately of 200 x 130 meters, while the path travelled by the human operator can be estimated in 1.450 meters (without considering the repeated passages on the same roads). 7
Laser Scanner IMU (helmet)
Backpack with the PC
Smartphone for the online visualization of the map IMU (waist) Figure 2: Human operator and wearable sensors.
During the path, fourteen loops have been closed, as it can be seen in Figure 3. In the following, with the term loop, we always mean inner loops, i.e., we ignore bigger loops that can include smaller loops inside them. 175
An overall description of the proposed dataset is given in Table 1. The dataset is shared both as comma-separated values (csv) files and as ROS file, exploiting the architecture based on ROS topics [17]. Indeed, ROS has become a standard in robotics research and facilitates integration of contributions by various researchers. Moreover, a great amount of the most common algorithms for SLAM
180
purposes are already shared as ROS applications. Thus, the dataset has been also saved as a bag file, a format for storing ROS message data, widely used from researchers to record and store datasets. Going more into details, the structure of the dataset is the following: • Laser Data: direct acquisition of the laser scanner, at 40 Hz. The laser scanner
185
returns 720 laser range data every 25 milliseconds, with and angular resolution of 0.375 degrees (from -135 to +135 degrees), and a maximum range of 30m. 8
Figure 3: Area where the dataset has been acquired. The red lines indicate the path followed by the human operator. Image taken with Google Earth.
The ROS message is encoded as a laser scan message, while each line of the csv file is structured with a timestamp and the 720 ranges of the laser (Table 1, row I). 190
• IMU (head) data: raw data of the IMU board placed on the helmet at 125 Hz. Data related to the acceleration, the angular velocity and the heading on the
three axes are saved every 8 milliseconds. The ROS data are encoded with a custom ROS message, which has the same structure of the csv file: timestamp, acceleration (x, y, z), angular velocity (x, y, z), orientation (x, y, z) (Table 1, row 195
II). • IMU (waist) data: raw data of the IMU board placed on the waist at 125 Hz. The
structure of the ROS message and of the csv file is the same as before (Table 1, row III).
While these three information have been directly acquired with the described set of 9
Laser data
Frequency (Hz)
Number of Data
CSV File
40
83945
laserscan.csv
CSV Data Format Timestamp (ns), Scan Ranges [0 to 719] (m).
ROS topic
Ros message type
/scan
sensor msgs/laser scan
/imudata
custom msgs/imu
/imudata
custom msgs/imu
/base scan
sensor msgs/laser scan
/odompose
geometry msgs/Twist
/headpose
geometry msgs/Twist
Timestamp (ns), IMU (head)
125
261487
Acceleration [x, y, z] (m2 /s),
imu head.csv
Orientation [x, y, z] (rad), Angular velocities [x, y, z] (m/s). Timestamp (ns),
IMU (body)
125
261497
Acceleration [x, y, z] (m2 /s),
imu body.csv
Orientation [x, y, z] (rad), Angular velocities [x, y, z] (m/s).
Laser data (synchronized with the
Timestamp (ns),
40
83904
laserscan.csv
125
254555
odom body.csv
Scan Ranges [0 to 719] (m).
Odometry) Odometry (head) Odometry (body)
Timestamp (ns), Position [x, y] (m), Heading [θ ] (rad). Timestamp (ns), 125
254565
odom head.csv
Position [x, y] (m), Heading [θ ] (rad).
Table 1: Data format of the presented dataset
200
wearable sensors, the Zena dataset has been also integrated with the human odometry, online estimated by means of the IMU data. Indeed, the almost totality of SLAM algorithms requires an estimation of the odometry (of the robot or, in this case, of the human operator), and therefore the information coming from the IMU boards should be processed in order to obtain an estimation of the position of the user. Additionally, this
205
processing should be done online, giving the Search&Rescue context and the necessity of generating a map while the human operator is walking. In the current approach, the estimation of the human odometry has been performed using a step detection procedure. This procedure is described in [16] and it is based on an algorithm for peaks (steps) discovering. The acceleration measured by the IMU
210
is analysed and, assuming that there is a periodic pattern in the signal, the steps are detected. Briefly, the accelorometer raw signal is processed with a low-pass filter, in order to remove high frequencies components, and after that, the vertical acceleration data are fed to a standard signal analysis algorithm to identify the peaks2 . Human ori2 The
peak
identification
procedure
is
available
10
at:
http://www.mathworks.com/
entation and step directions are determined using the compass integrated in the IMU 215
boards (filtered). The step length can depend on different factors among which the user’s height, but scan matching algorithms used in common SLAM algorithms well compensate the movement and estimate the user’s position at each step (same considerations for the orientation). In our model, the step has been fixed to 0.6 meter, but tests have been performed varying this parameter (results will be shown in Section 4).
220
For the current purposes, this approach has been implemented online, using temporal windows, in which the signal is analysed and the steps are detected. The estimation of the optimal window length is not trivial. At this aim, different window lengths have been tested: it has been noticed that with a window length greater than one second, steps were always detected with a good approximation. Indeed, an offline analysis of
225
the complete raw signal of the accelerometer detects a comparable number of steps. The algorithm estimates the user’s motion analyzing the acceleration pattern as shown in Figure 4. When a step is detected, the motion of the user is estimated with a step forward, the motion is spread between the previous spike and the detected one and the information is integrated with the heading (measured with the compass) in order
230
to determine the step direction. It can be seen how the greater is the chosen temporal window, the higher is the difference between the real motion of the operator and the estimated one: indeed, the step is interpolated from the start of the window (or from the previous spike) to the end of the step, and in the case of a single step in a wide temporal window, substantial errors can arise. For this reason, a temporal window length of one
235
second (the minimum possible length that allows a precise detection of the step) has been chosen. Finally, odometry data should be synchronized with the laser ranges information, i.e. laser data should be delayed of a temporal window in order to be used together with the online estimated odometry. Therefore, the dataset has been enriched with the following data:
240
• Human Odometry (head): estimation of the odometry using the accelerometer and the compass mounted on the head. The x and y position and the orientation
matlabcentral/fileexchange/25500-peakfinder
11
吀攀洀瀀漀爀 愀氀 眀椀 渀搀漀眀
吀攀洀瀀漀爀 愀氀 眀椀 渀搀漀眀
Figure 4: Accelerometer data (up) and estimation of the human odometry. For the sake of clearness, in this case the motion has been estimated all along the x axis. The red lines correspond to the end of the step detection.
θ along the z axis are estimated each 8 ms (with the spreading process shown before). The ROS message is encoded as a twist message, while each line of the csv file is structured with a timestamp, the x and y position, and the heading θ 245
(Table 1, row V). • Human Odometry (body): estimation of the odometry using the accelerometer and the compass mounted on the waist. The spreading process, and the structure of the ROS message and of the csv file is the same as before (Table 1, row VI). • Laser scan (synchronized with the odometry): laser scanner data delayed of one
250
second (the odometry temporal window). The information is already contained in the Laser Data csv file; however, for helping the final users, an additional ROS topic has been added to the bag file (Table 1, row IV).
12
The whole dataset, including all csv files and the ROS bag file, is made available at: https://github.com/laboratoriumunige/zenadataset. 255
It is worth saying that the process of estimating human odometry is inevitably affected by a certain error: this is due to the fixed step length, to errors in the step detection phases, and also to the fact that the compass measurement is often affected by buildings’ structures, such as steel frame, iron pipes. However, as it will shown in the following paragraph, estimating the position of the user with the proposed approach
260
allow the SLAM algorithm to reconstruct a reliable map. 4. Analysis This final section aims at giving a preliminary validation of the proposed dataset, using it as input of a commonly used algorithm for robotics SLAM, in order to give some starting references to the users and to investigate some aspects of the dataset itself
265
and of the chosen set of wearable sensors. The Zena dataset has been investigated by using the KartoSLAM algorithm [18]. It must be here reminded that the utilization of the KartoSLAM algorithm is possible only if there is an estimation of the odometry of the robot (or in this case of the human operator), since it requires the pose and orientation of the body and laser frames with
270
respect to the world frame. Thus, the odometry estimated as shown in the previous section has been used as input. The KartoSLAM package is a robust algorithm for Localization and Mapping, based on the graph-SLAM technique. In the graph-SLAM approach the measurements are represented as a graph, in which every node represents a local map, and it is la-
275
beled with the location at which the measurement was taken, and the edges between two nodes encode the spatial information arising from the alignment of the connected measurements (they can be seen as the spatial constraints between two nodes). The problem is then twofold: identify the constraints based on sensor data, and correct the poses of the robot to obtain a consistent map of the environment. There are differ-
280
ent strategies aimed at solving these problems: among the most relevant, the Sparse Pose Adjustment (SPA) and the G2 O (General Graph Optimization) are adopted in
13
the KartoSLAM approach. These methods follow the concept of bundle-adjustment (BA) algorithms originally developed for computer vision, exploiting the sparsity of the measurement graph so as to speed up computation. 285
In the proposed work, the SPA approach has proved to be the more reliable one, in relation to the chosen sensors worn by the human operator, and to the structure of the environment, and it has therefore been used during the analysis of the dataset. A preliminary work has been performed for the calibration of the parameters, in order to find a configuration that allowed to profitably complete the mapping task; in particular
290
the following parameters have been adapted for the particular outdoor context: • range threshold: it sets the scanning range considered by the algorithm for scan matching. It has been set to 8 meters.
• minimum travel distance: it is the minimum travel distance between scans, i.e.
the mapper will use the data from the new scan only if a new scan’s position is
295
more than this value from the previous scan. It has been set to 0.5 m (default value was 0.2 m) • minimum travel heading: this parameter sets the minimum heading change, i.e.
the mapper will use the data from the new scan only if a new scan’s heading is more than this value from the previous scan. It has been set to 0.06 rad (default
300
value was 0.549 rad) • correlation search space dimension: the size of the search grid used by the matcher when matching sequential scans. It has been set to 1.0 m (default value is 0.3 m). The KartoSLAM algorithm has been applied to the dataset directly during the ac-
305
quisition of the data, in order to test the capacity of the system of generating a map online, a crucial aspect in a Search&Rescue scenario. However, because of the GraphSLAM structure, the number of nodes and the size of the graph increase with the user’s motion, and after some minutes the map can not be easily handled in real-time (i.e, with no visible delays between sensor acquisition and mapping). Moreover, the size of
14
310
the map that can be handled strictly depends on the onboard processing power. Processing the current dataset online with the notebook used for the acquisition (i.e., Asus Zenbook UX501VW with a Intel Core i7-6700HQ) the KartoSLAM algorithm can generate a valid map for around 10 minutes, closing five loops. After this point the number of nodes of the graph describing the total map becomes too high (around 2000
315
nodes) and the corresponding matrices can not be correctly handled in real-time. The same dataset has been tested with a different CPU, a Intel Core i7-6700K, and in this case the algorithm is able to generate a valid and reliable online map for 990 seconds, closing seven loops and generating 2934 nodes. Figure 5 shows the portion of the map than can be generated online with the described hardware.
Figure 5: Biggest map obtained processing the data online. Seven loops are closed, and 2934 nodes are generated; after 990 seconds, the size of the graph is too big to be processed online, and errors arise in the mapping process.
320
During the acquisition, the user can monitor the map building process by using a custom-built android application, that exploiting the ROS-for-android version, connects to the map topic (generated by the kartoSLAM algorithm running on the notebook) and may display the map on a common smartphone. A screenshot of the application is shown in Figure 6.
15
Figure 6: Screenshot of the android application for online map visualization
325
In order to process the whole dataset, and thus obtain some insights about the role of each sensor (belt IMU, head-mounted IMU, head-mounted laser scanner) for producing accurate maps and on the precision of the human odometry estimation, the dataset has been also processed offline with the KartoSLAM algorithm. First of all, the dataset has been tested in relation to the set of wearable sensors
330
used for the acquisition: in particular, the advantages arising from using two IMUs (on the head and on the waist) have been evaluated. Indeed, as mentioned before, the KartoSLAM algorithm requires two different mobile frames: • Frame body, that describes the human body’s position and orientation on a plane. • Frame laser, that describes the position and orientation of the laser on a plane.
335
The utilization of these frames implies the ignoring of 3D position and attitude, thus approximating the system as if it were a wheeled robot moving on a flat ground. This is a strong assumption that, however, proves to be acceptable during experiments performed in a urban settings, where we are more interested in detecting free spaces 16
and occupied spaces determined by buildings, than in drawing an accurate 3D repre340
sentation. Tests have been performed with these frames in different configurations as described in the following: Configuration a: • x, y and θ of body frame are measured through the human odometry approach
345
described in Section 3, using the IMU located in the belt (i.e. x and y and θ from
the Human Odometry (body) of the dataset); • x, y of laser frame are the same of body frame (plus a constant offset); • θ of laser frame is measured using the compass located on the helmet (the heading θ from the Human Odometry (head) of the dataset).
350
Configuration b: • x, y and θ of body frame and x, y of laser frame are the same of Configuration a; • θ of laser frame is assumed to be the same measured by the compass located on the waist, i.e., θ of laser frame = θ of body frame.
In other words, this configuration corresponds to the frames description that could be 355
done using a single IMU located on the waist (i.e., only data from the Human Odometry (body) of the dataset). Configuration c: • x, y and θ of laser frame are calculated using the IMU located in the helmet (i.e., x and y and θ from the Human Odometry (head) of the dataset);
360
• x, y of body frame are the same of laser frame (plus a constant offset); • θ of body frame is assumed to be the same measured by the compass located on the helmet (the heading θ from the Human Odometry (head) of the dataset).
In other words, this configuration corresponds to the frames description that could be done using a single IMU located on the helmet (i.e., only data from the Human 365
Odometry (head) of the dataset). 17
The percentage of loops closed, averaged over five repeated runs, using these different frames configuration has been considered as performance indicator. The repetition of the runs is necessary since the KartoSLAM algorithm has stochastic components related to the handling of the odometry and to the scan matching algorithm; therefore, 370
the maps obtained, with the same inputs and parameters, can slightly differ from each other. As expected, the Configuration a gives the best results, allowing, at each run, to close almost the totality of the loops and obtaining a reliable map, since it takes into account the orientation of the head with respect of the body. On the other side, Config-
375
uration b allows the algorithm to close on average only the 39.29% of loops, making a wrong correlation between the laser scan data (in the head frame) with the body frame, losing precision in the scan matching process. Finally, Configuration c allows, on average, for the closure of the 57.14% of loops, neglecting the kinematic chain of the human body, with errors in the calculation of its position and orientation. Indeed,
380
even if humans are potentially able to move in all directions on the xy plane (i.e., not depending on the heading of their body and of their head), nonetheless the preferred direction of motion is usually forward. Then, treating them as a unicycle robot with a panning laser appears to be more convenient than modelling them as a holonomous robot.
385
Figure 7 shows two examples of the map calculated by using frames in Configuration a and in Configuration c.
18
1
2 1
3 4 5
6
8
7
9
11
2
10
3
12
4
5
13
14
6
Figure 7: On the left, map obtained using Configuration a. All 14 loops are closed. On the right, map obtained using Configuration c. Only 6 loops are closed, and the map building process is not reliable.
The second aspect analized is strictly related to the process of the step detection. Indeed, it has been described how the step detection process assumes that the step length is equal to 0.6 meter. This has been estimated in relation to the average walking 390
pattern of the user, but obviously the step detection and the mapping processes should be robust in relation to the variation of this parameter. To this aim, the dataset was tested (with frames in Configuration a) while varying the value of the step, from 0.4 to 1.0 meter. Also in this case, the values are averaged over five runs. It can be seen how the variation of the step size has some influences only if the step size goes below
395
a certain value (the percentage of loop closures is below 75% if the step size is 0.4m); otherwise, the percentage of loop closures is always higher than 90%, showing that the step detection process is robust to the variation of the step size. Table 2 summarizes the results obtained in the analysis of the dataset.
19
Frame Configuration Step size(m) Percentage of loop closures
a
b
c
a
a
a
a
a
a
0.6
0.6
0.6
0.4
0.5
0.7
0.8
0.9
1.0
97.14 %
39.29%
57.14%
71.40%
91.43%
97.14%
95.70%
94.28%
92.86%
Table 2: Percentage of loop closures of the maps, with different frames configurations and different step sizes.
5. Conclusions 400
While the possibility for first responders and human operators of mapping an unknown scenarios while walking can be of the utmost importance in Search&Rescue scenarios, this research subject has not been so much taken under investigation from the research community. As a consequence, there are few available datasets related to this topic, and they are all limited to indoor environments.
405
The presented article describes the acquisition procedure of a 35-minutes dataset for human SLAM in a complex outdoor scenario, such as the historical centre of Genova. A laser scanner and two IMUs have been used as wearable sensors. For the estimation of the human odometry, a procedure for the online detection of the step, using the two IMU boards, has been implemented.
410
The analysis of the Zena dataset, using a popular algorithm for SLAM specifically tailored for mobile robots, shows how the proposed procedure allows reconstructing the map of the surrounding environment, in particular when using all data of the wearable sensors, i.e., two different frames for the laser position (mounted on the helmet of the user) and for the body position. Finally, the step detection procedure has been proved
415
robust to variations of the step size. Future works will be focused in integrating strategies for long-term maintenance of the graph [19] [20], to avoid the real-time issues arising when processing the dataset online.
20
6. References 420
[1] Durrant-Whyte, H., & Bailey, T. (2006). Simultaneous localization and mapping: part I. IEEE robotics & automation magazine, 13(2), 99-110. [2] Cinaz, B., & Kenn, H. (2008, September). HeadSLAM-simultaneous localization and mapping with head-mounted inertial and laser range sensors. In 2008 12th IEEE International Symposium on Wearable Computers (pp. 3-10). IEEE.
425
[3] Zlot, R., Bosse, M., Greenop, K., Jarzab, Z., Juckes, E., & Roberts, J. (2014). Efficiently capturing large, complex cultural heritage sites with a handheld mobile 3D laser mapping system. Journal of Cultural Heritage, 15(6), 670-678. [4] Newman, P., & Corke, P. (2009). Editorial Data Papers-Peer Reviewed Publication of High Quality Data Sets. The International Journal of Robotics Research,
430
28(5), 587-587. [5] Stachniss, C., Frese, U., & Grisetti, G. (2007). Openslam. org. [6] Simpson, R., Cullip, J., & Revell, J. (2011). The cheddar gorge data set. Tech. Rep. [7] Wang, C. C., Duggins, D., Gowdy, J., Kozar, J., MacLachlan, R., Mertz, C., ...
435
& Thorpe, C. (2004). Navlab slammot datasets. Pittsburgh, PA: Carnegie Mellon University. [8] http://www.nada.kth.se/∼johnf/kthdata/dataset.html. Online, January 2017 [9] Howard, A., & Roy, N. (2003). The robotics data set repository (radish). [10] Stachniss, C. (2006). Exploration and mapping with mobile robots (Doctoral dis-
440
sertation, University of Freiburg). [11] Blanco, J. L., Moreno, F. A., & Gonzalez, J. (2009). A collection of outdoor robotic datasets with centimeter-accuracy ground truth. Autonomous Robots, 27(4), 327-351.
21
[12] Kohlbrecher, S., Von Stryk, O., Meyer, J., & Klingauf, U. (2011, November). A 445
flexible and scalable slam system with full 3d motion estimation. In 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics (pp. 155-160). IEEE. [13] https://code.google.com/archive/p/tu-darmstadt-ros-pkg/. Online, January 2017 [14] Sibley, G., Mei, C., Reid, I., & Newman, P. (2010). Vast-scale outdoor navigation
450
using adaptive relative bundle adjustment. The International Journal of Robotics Research, 29(8), 958-980. [15] https://data.csiro.au/dap/. Online, January 2017 [16] Bruno, B., Mastrogiovanni, F. & Sgorbissa, A. (2015). HOOD: a Real Environment Human Odometry Dataset for Wearable Sensor Placement Analysis. In In-
455
telligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on, At Hamburg, Germany. [17] Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., ... & Ng, A. Y. (2009, May). ROS: an open-source Robot Operating System. In ICRA workshop on open source software (Vol. 3, No. 3.2, p. 5).
460
[18] Santos, J. M., Portugal, D., & Rocha, R. P. (2013, October). An evaluation of 2D SLAM techniques available in robot operating system. In 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) (pp. 1-6). IEEE. [19] Walcott-Bryant, A., Kaess, M., Johannsson, H., & Leonard, J. J. (2012, October). Dynamic pose graph SLAM: Long-term mapping in low dynamic environments.
465
In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1871-1878). IEEE. [20] Einhorn, E., & Gross, H. M. (2015). Generic NDT mapping in dynamic environments and its application for lifelong SLAM. Robotics and Autonomous Systems, 69, 28-39.
22
Carmine Tommaso Recchiuto received his Master Degree in Electronic Engineering from the University of Pisa in 2008. From 2009 to 2012 he was Research Fellow at the Scuola Superiore Sant’Anna, where he was involved in projects related to the realization of tactile sensors and humanoid robotics. From 2013 he was Research Fellow in the Department of Informatics, Bioengineering, Robotics, and System Sciences (DIBRIS), at the University of Genova, where he was involved in the PRISMA project, focussing his work in the implementation of UAVs in emergency scenarios. He received his Ph.D in 2015 in Microsystems from the University of Rome Tor Vergata. He is currently Post-Doc Fellow with the University of Genova. His main interests are in the fields of tacitle sensors, UAVs, humanoids and human-robot interaction.
Antonello Scalmato received the M.S. degree in computer science engineering and the Ph.D. degree from the University of Genoa, Genoa, Italy, in 2008 and 2011, respectively. He is a Research Fellow with the Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa. His research interests include ontology applications for mobile robotics and ambient intelligence. His current research interests include context-aware systems for smart homes and robots in intelligent environments.
Antonio Sgorbissa is Associate Professor at University of Genova, Department of Informatics, Bioengineering, Robotics, and Systems Sciences (DIBRIS), Italy. In 2000 he received his Ph.D. in Robotics from the University of Genova, and from 2001 to 2004 he was a Post Doc at Georgia Tech, University of Parma and later at University of Genova. He currently teaches Ambient Intelligence, Cooperative Robotics, and Real-Time Operating Systems at the Faculty of Engineering as well as in the European Master on Advanced Robotics (EMARO), and Computer Science at the Faculty of Psychology. His main research interests are in the area of multi-robot systems, wearable robotics, and ambient intelligence. His research interests include also planning, knowledge representation, and machine consciousness. He is author/coauthor of more than 100 international scientific papers, and of two international patents in Robotics
Highlights - A novel dataset acquired by human operators endowed with wearable sensors -
Wereable sensors are placed taking into account the kinematic constraints of the human SLAM problem
-
Human odometry is based on a step estimation process
-
The analysis of the dataset shows the importance of estimating both head and body frames, and the robustness of the step detection process.