MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark
CVPR'24
News
2024.06.26: We release the dataset under limited conditions. Please refer to the bottom for detail.
2024.03.25: Hello, world! Our MTMMC site opened.
Overview
Abstract
Multi-target multi-camera tracking is a crucial task that involves identifying and tracking individuals over time using video streams from multiple cameras. This task has practical applications in various fields, such as visual surveillance, crowd behavior analysis, and anomaly detection. However, due to the difficulty and cost of collecting and labeling data, existing datasets for this task are either synthetically generated or artificially constructed within a controlled camera network setting, which limits their ability to model real-world dynamics and generalize to diverse camera configurations. To address this issue, we present MTMMC, a real-world, large-scale dataset that includes long video sequences captured by 16 multi-modal cameras in two different environments - campus and factory - across various time, weather, and season conditions. This dataset provides a challenging test bed for studying multi-camera tracking under diverse real-world complexities and includes an additional input modality of spatially aligned and temporally synchronized RGB and thermal cameras, which enhances the accuracy of multi-camera tracking. MTMMC is a super-set of existing datasets, benefiting independent fields such as person detection, re-identification, and multiple object tracking. We provide baselines and new learning setups on this dataset and set the reference scores for future studies. The datasets, models, and test server will be made publicly available.
MTMMC
3D Layout Overview of the Camera Positions
(a) Campus / (b) Factory. A total of 16 cameras are densely installed in the buildings. The cameras provide both the RGB and thermal data that are spatially and temporally aligned.
3D Layout Overview of the Camera Positions
Key Characteristics of MTMMC
Realistic Capturing Environment
Diverse Sites: Campus & Factory
Realistic Camera Topology: Indoor, Outdoor, Multiple Floors, Overlapping Views
Multi-modality
RGB and Thermal
Diversity
Diverse Track: track length(visited camera; 1~16), track scale
Diverse Condition: time(day~evening), weather(sunny, cloudy), seasons(summer, fall)
Multi-modal Images of Each Camera
Details
# Cameras: 16 / # ID: 3,669 / # Frames: 3,052,800
Camera Coverage: indoor & outdoor
Modalities: RGB and Thermal
Resolution: 1920x1080
DOWNLOAD
Data
We provide the video frames and annotations:
Frames: JPG format
Annotations: COCO style JSON format
You can request access to the dataset by contacting us (POC: dlsrbgg33@kaist.ac.kr).
Please include a completed copy of the dataset terms and conditions form (link) with your information.
Code
The code and models of the baseline methods will be released.
Acknowledgment
We gratefully acknowledge that MTMMC was built as part of the AI Training Data Construction Project 2021 hosted by the Ministry of Science and ICT (MSIT) and supported by the National Information Society Agency (NIA) of South Korea.