MuSe dataset Repository

For benchmarking SLAM in challenging scenarios

Abstract

Simultaneous localization and mapping (SLAM) is one of the core building blocks of autonomous systems. For a robot equipped with onboard sensors, SLAM entails the joint estimation of the state of the robot and a representation (map) of the environment using the perceptual information from the sensors. State-of-the-art SLAM algorithms often fail when the sensors produce unreliable information due to malfunctioning, aging, or off-normal environmental conditions. In practice, however, such conditions are not rare and arise in various indoor scenarios. Further, most existing SLAM algorithms fail when the environment also contains dynamic features, such as those arising from moving objects or people. Fusion of information from multiple sensors can not only help improve the accuracy of the estimates but also make the system more robust to failures. In this work, we recreate such situations in a dataset to push research towards solving and benchmarking SLAM in challenging conditions. The proposed Multi-Sensor (MuSe) dataset contains exigent scenarios faced by a two-wheel differential drive robot equipped with a multi-sensory setup navigating in an indoor environment along with ground truth information for benchmarking state-of-the-art SLAM solutions.

Motivation

The importance of developing robust SLAM systems for challenging environments is necessary for widespread usability of autonomous mobile robots. This is a recurring theme at the recent flagship robotic conferences [20]. In contrast to the number of solutions that have come up recently for this, there are very few comparative studies to evaluate these techniques on common ground. This research void is primarily due to the lack of datasets that provide challenging grounds for benchmarking these solutions. The MuSe dataset caters to the development and benchmarking of such solutions.

MuSe is directed towards benchmarking of solutions to some of the open problems in SLAM

For detailed overview of the MuSe dataset refer to the full text here

Existing Datasets for SLAM

Table 1: Existing Indoor Datasets and sensors for SLAM

MuSe is unique and contains

Data from diverse sensor suite felicitating sensor fusion

Accurate ground-truth information for benchmarking

Challenging scenarios where state-of-the art SLAM fails

MuSe would be of interest to researchers studying open problems in SLAM

MuSe Setup

Figure: Robot with sensor setup

Table 2: List of sensors used in MuSe dataset

System setup for multi-sensor data collection

The data collection setup consists of 3 computers:

1) Mobile base NVIDIA Jetson Tx2 connected to the sensors mounted on the robot, running Ubuntu 16.04 ROS Kinetic2) The base station computer running Ubuntu 16.04 and ROS Kinetic.3) The Vicon camera host PC running Windows 10 and the Vicon Tracker software

The Vicon camera host system process the feed from Vicon cameras and sends the ground-truth information to the base station which relays it to the mobile base computer over the WiFi network where it is recorded along with other sensory data.

Testbed and Scenarios

Different from the existing datasets, our MuSe Repository contains multi-sensor data collected in an 12x6 m² indoor environment equipped with a seven camera Vicon motion capture system with the following settings:

Regular Scenario

Well illuminated feature-rich conditions designed for benchmarking existing solutions

Sensor complementary scenario

Contains instances when information from a subset of sensors gets compromised. Designed to test the robustness of the SLAM solutions towards sensor failures

Dynamic scenario

Contains situations with moving people and mutating environmental features. Designed to evaluate SLAM solutions in non-rigid environments

Open Problems in SLAM that MuSe caters to

The MuSe dataset repository provides challenging grounds for the development of SLAM solutions to some of the open problems in the area [1], a few of these challenges are listed below.:

How to fuse multi-sensor information so as to exploit the complementary strengths of various modalities?
How to represent map using multi-sensor information?
Can SLAM algorithms detect sensor degradation?
How to resolve conflicting sensor information?
Due to reduced quality of sensor measurements the sensor noise model changes. How to adjust sensor noise accordingly?
Can SLAM algorithms capture the environment dynamism? How to represent maps that change over time?
Can multi-sensor information help identify and track dynamic objects?

Data Collection Details

Table 3: Dataset characteristics

Data was collected in 5 sessions with various scenarios, the Table here shows the names and summary of each collection session

Robot ground truth trajectories from all data collection sessions, ground truth positions of range-only sensor anchors

Dataset Format

rosbag data format

We recorded the dataset using rosbag, the standard tool for data recording provided by ROS. The resulting so-called bag files (*.bag) contains recorded data from all sensors along with ground truth from the Vicon system. For each run of the robot we collected a bag file containing data from each sensor in the form of Message, a simple data structure,comprising typed fields. Data from each sensor in the form of Message can be accessed over a Topic, a named bus to exchange information in ROS framework. For portability and ease of usage the rosbag file for each run is split into 5GB chunks. The table below shows the published ROS Topics for various sensors.

Table 4: List of data Topics, human readable files, and field names for all sensors in various formats for MuSe dataset

human readable format

Data is available in human readable format as well. This will be helpful for users interested in the dataset and are not familiar with ROS framework. Here we briefly describe the organisation. Data for each run of the robot is split into various chunks. Each sensor system in a data chunk eg. RPlidar, kinect, zed camera, myAHRS, etc. has its own folder. A sensor system may have multiple modalities, each modality is a sub-folder within the sensor system folder. These sub-folders contain human readable text files and optionally a folder containing timestamped images (only for image modality). An overview of the folder hierarchy for a data chunk is illustrated below.

MuSe dataset provides data from a diverse multi-sensor sensor suite having ten different modalities. Data for each modality (except for images) of a sensor system is stored in JSON format in a text file. For image data from each sensor of the sensor system, a JSON format text file containing timestamps of all images collected along with a folder containing timestamped images in png format is given. Depth images from kinect are accompanied with a numpy array in a ∗.npy file containing depth information. Detailed documentation on the format of the JSON encoded text files can be found here.

matlab data format

For MATLAB users, the dataset was imported into MATLAB workspace and made available in ∗.mat format files accompanied with camera images. The ∗.mat files for each dataset contains a MATLAB structure array (struct) incorporating information from sensors except for the images from the vision sensors which are provided in a separate folder. Each field of the struct contains information from a sensor in a MATLAB table data structure (these are often used for storing heterogeneous data and metadata into a single container). The table above shows the filed names to access for each sensor. The tabled data for each sensor contains multiple variables such as timestamps and other sensor specific data fields similar to ROS. More details on the same and example usage files are provided. The folder structure for the camera data accompanied with MATLAB format data is shown below. Detailed documentation and examples of using the MATLAB format can be found here.

Download

Calibration Files

Sample Data

Full Data

Software tools to access the data

Matlab Tools

Access Complete Dataset

References

[1] Cadena, Cesar, et al. "Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age." IEEE Transactions on robotics 32.6 (2016): 1309-1332.[2] A. Pronobis and B. Caputo, “COLD: The CoSy Localization Database,” The International Journal of Robotics Research (IJRR), vol. 28, no. 5, pp. 588–594, 2009.[3] M. Fallon, H. Johannsson, M. Kaess, and J. J. Leonard, “The mit stata center dataset,” The International Journal of Robotics Research (IJRR), vol. 32, no. 14, pp. 1695–1699, 2013.[4] C. Stachniss, et al. “Photogrammetry Lab Datasets,” accessed dec-2018. [Online]. Available: http://www.ipb.uni-bonn.de/datasets/[5] L. Carlone, “Pose graph optimization datasets,” accessed dec-2018, MIT. [Online]. Available: https://lucacarlone.mit.edu/datasets/[6] Handa, Ankur, et al. "A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM." 2014 IEEE International conference on Robotics and automation (ICRA). IEEE, 2014.[7] Schubert, David, et al. "The TUM VI benchmark for evaluating visual-inertial odometry." 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018.[8] D. Caruso, et al. “Large-scale direct slam for omnidirectional cameras: Dataset,” 2015, omnidirectional dataset. [Online]. Available: https://vision.in.tum.de/data/datasets/omni-lsdslam[9] J. Djugash, B. Hamner, and S. Roth, “Navigating with ranging radios: Five data sets with ground truth,” Journal of Field Robotics (JFR), vol. 26, no. 9, pp. 689–695, 2009.[10] F. Pomerleau, M. Liu, F. Colas, and R. Siegwart, “Challenging data sets for point cloud registration algorithms,” The International Journal of Robotics Research (IJRR), vol. 31, no. 14, pp. 1705–1711, 2012. [11] M. Burri, J. Nikolic, P. Gohl, T. Schneider, J. Rehder, S. Omari, M. W. Achtelik, and R. Siegwart, “The EuRoC Micro Aerial Vehicle Datasets,” The International Journal of Robotics Research (IJRR), 2016.[12] Sturm, Jürgen, et al. "A benchmark for the evaluation of RGB-D SLAM systems." 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012.[13] A. Handa, T. Whelan, J. McDonald, and A. J. Davison, “A Benchmark for RGBD Visual Odometry, 3D Reconstruction and SLAM,” in Proc. IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2014, pp. 1524–1531.[14] O. Wasenmuller, M. Meyer, and D. Stricker, “CoRBS: Comprehensive RGB-D benchmark for SLAM using Kinect v2,” in IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2016.[15] D. Schubert, T. Goll, N. Demmel, V. Usenko, J. St¨uckler, and D. Cremers, “The tum vi benchmark for evaluating visual-inertial odometry,” arXiv preprint arXiv:1804.06120, 2018.[16] Rawseeds, “Rawseeds,” accessed dec-2018. [Online]. Available: http://www.rawseeds.org/home/[17] Hess, Wolfgang, et al. "Real-time loop closure in 2D LIDAR SLAM." IEEE International conference on Robotics and automation (ICRA) 2016.[18] Labbé, Mathieu, et al. "RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation." Journal of Field Robotics (JFR) 2019.[19] Engel, Jakob, Vladlen Koltun, and Daniel Cremers. "Direct sparse odometry." IEEE Transactions on Pattern Analysis and Machine Intelligence 2018.[20] Kramer A, Robust Perception For Autonomous Field Robots in Challenging Environments ICRA 2021 Workshop. URL http://robustperception.net/. [Online; accessed May 2021].

Google Sites

Report abuse