MuSe dataset Repository
For benchmarking SLAM in challenging scenarios
Abstract
Simultaneous localization and mapping (SLAM) is one of the core building blocks of autonomous systems. For a robot equipped with onboard sensors, SLAM entails the joint estimation of the state of the robot and a representation (map) of the environment using the perceptual information from the sensors. State-of-the-art SLAM algorithms often fail when the sensors produce unreliable information due to malfunctioning, aging, or off-normal environmental conditions. In practice, however, such conditions are not rare and arise in various indoor scenarios. Further, most existing SLAM algorithms fail when the environment also contains dynamic features, such as those arising from moving objects or people. Fusion of information from multiple sensors can not only help improve the accuracy of the estimates but also make the system more robust to failures. In this work, we recreate such situations in a dataset to push research towards solving and benchmarking SLAM in challenging conditions. The proposed Multi-Sensor (MuSe) dataset contains exigent scenarios faced by a two-wheel differential drive robot equipped with a multi-sensory setup navigating in an indoor environment along with ground truth information for benchmarking state-of-the-art SLAM solutions.
Motivation
The importance of developing robust SLAM systems for challenging environments is necessary for widespread usability of autonomous mobile robots. This is a recurring theme at the recent flagship robotic conferences [20]. In contrast to the number of solutions that have come up recently for this, there are very few comparative studies to evaluate these techniques on common ground. This research void is primarily due to the lack of datasets that provide challenging grounds for benchmarking these solutions. The MuSe dataset caters to the development and benchmarking of such solutions.
MuSe is directed towards benchmarking of solutions to some of the open problems in SLAM
Existing Datasets for SLAM
Table 1: Existing Indoor Datasets and sensors for SLAM
MuSe is unique and contains
Data from diverse sensor suite felicitating sensor fusion
Accurate ground-truth information for benchmarking
Challenging scenarios where state-of-the art SLAM fails
MuSe would be of interest to researchers studying open problems in SLAM
MuSe Setup
Figure: Robot with sensor setup
Table 2: List of sensors used in MuSe dataset
System setup for multi-sensor data collection
The data collection setup consists of 3 computers:
The Vicon camera host system process the feed from Vicon cameras and sends the ground-truth information to the base station which relays it to the mobile base computer over the WiFi network where it is recorded along with other sensory data.
Testbed and Scenarios
Different from the existing datasets, our MuSe Repository contains multi-sensor data collected in an 12x6 m2 indoor environment equipped with a seven camera Vicon motion capture system with the following settings:
Regular Scenario
Well illuminated feature-rich conditions designed for benchmarking existing solutions
Sensor complementary scenario
Contains instances when information from a subset of sensors gets compromised. Designed to test the robustness of the SLAM solutions towards sensor failures
Dynamic scenario
Contains situations with moving people and mutating environmental features. Designed to evaluate SLAM solutions in non-rigid environments
Open Problems in SLAM that MuSe caters to
The MuSe dataset repository provides challenging grounds for the development of SLAM solutions to some of the open problems in the area [1], a few of these challenges are listed below.:
How to fuse multi-sensor information so as to exploit the complementary strengths of various modalities?
How to represent map using multi-sensor information?
Can SLAM algorithms detect sensor degradation?
How to resolve conflicting sensor information?
Due to reduced quality of sensor measurements the sensor noise model changes. How to adjust sensor noise accordingly?
Can SLAM algorithms capture the environment dynamism? How to represent maps that change over time?
Can multi-sensor information help identify and track dynamic objects?
Data Collection Details
Table 3: Dataset characteristics
Data was collected in 5 sessions with various scenarios, the Table here shows the names and summary of each collection session
Robot ground truth trajectories from all data collection sessions, ground truth positions of range-only sensor anchors
Dataset Format
rosbag data format
We recorded the dataset using rosbag, the standard tool for data recording provided by ROS. The resulting so-called bag files (*.bag) contains recorded data from all sensors along with ground truth from the Vicon system. For each run of the robot we collected a bag file containing data from each sensor in the form of Message, a simple data structure,comprising typed fields. Data from each sensor in the form of Message can be accessed over a Topic, a named bus to exchange information in ROS framework. For portability and ease of usage the rosbag file for each run is split into 5GB chunks. The table below shows the published ROS Topics for various sensors.
Table 4: List of data Topics, human readable files, and field names for all sensors in various formats for MuSe dataset
human readable format
Data is available in human readable format as well. This will be helpful for users interested in the dataset and are not familiar with ROS framework. Here we briefly describe the organisation. Data for each run of the robot is split into various chunks. Each sensor system in a data chunk eg. RPlidar, kinect, zed camera, myAHRS, etc. has its own folder. A sensor system may have multiple modalities, each modality is a sub-folder within the sensor system folder. These sub-folders contain human readable text files and optionally a folder containing timestamped images (only for image modality). An overview of the folder hierarchy for a data chunk is illustrated below.
MuSe dataset provides data from a diverse multi-sensor sensor suite having ten different modalities. Data for each modality (except for images) of a sensor system is stored in JSON format in a text file. For image data from each sensor of the sensor system, a JSON format text file containing timestamps of all images collected along with a folder containing timestamped images in png format is given. Depth images from kinect are accompanied with a numpy array in a ∗.npy file containing depth information. Detailed documentation on the format of the JSON encoded text files can be found here.
matlab data format
For MATLAB users, the dataset was imported into MATLAB workspace and made available in ∗.mat format files accompanied with camera images. The ∗.mat files for each dataset contains a MATLAB structure array (struct) incorporating information from sensors except for the images from the vision sensors which are provided in a separate folder. Each field of the struct contains information from a sensor in a MATLAB table data structure (these are often used for storing heterogeneous data and metadata into a single container). The table above shows the filed names to access for each sensor. The tabled data for each sensor contains multiple variables such as timestamps and other sensor specific data fields similar to ROS. More details on the same and example usage files are provided. The folder structure for the camera data accompanied with MATLAB format data is shown below. Detailed documentation and examples of using the MATLAB format can be found here.
