A Dataset for Open Vocabulary
Temporal Repetition Counting in Videos

Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Andrew Zisserman

Google DeepMind

[PDF Paper][OVR-Ego4D Dataset][OVR-Kinetics Dataset][Dataset Visualizer Colab][Bibtex]

Abstract

We introduce a dataset of annotations of temporal repetitions in videos. The dataset, OVR, contains annotations for over 72K videos, with each annotation specifying the number of repetitions, the start and end time of the repetitions, and also a free-form description of what is repeating. The annotations are provided for videos sourced from Kinetics and Ego4D, and consequently cover both Exo and Ego viewing conditions, with a huge variety of actions and activities. Moreover, OVR is almost an order of magnitude larger than previous datasets for video repetition. We also propose a baseline transformer-based counting model, OVRCounter, that can localise and count repetitions in videos that are up to 320 frames long.  The model is trained and evaluated on the OVR dataset, and its performance assessedwith and without using text to specify the target class to count. The performance is also compared to a prior repetition counting model. 

OVRCounter

Bibtex

@misc{dwibedi2024ovrdatasetopenvocabulary,

      title={OVR: A Dataset for Open Vocabulary Temporal Repetition Counting in Videos}, 

      author={Debidatta Dwibedi and Yusuf Aytar and Jonathan Tompson and Andrew Zisserman},

      year={2024},

      eprint={2407.17085},

      archivePrefix={arXiv},

      primaryClass={cs.CV},

      url={https://arxiv.org/abs/2407.17085}, 

}