Call for Datasets
This call is now closed- stay tuned for the release of this dataset in Summer 2024!
Machine learning models have made great progress in automating the study of behavior; however, these models are brittle and often break down when applied to data from different settings and labs. To develop models that generalize across labs, we need datasets across labs for both training as well as evaluation on real-world changes in setting, pose annotation, video quality, and behavior.
We are working to create a large cross-lab benchmark dataset on mouse social behavior, to benchmark the generalization ability of current models for behavior analysis. We would welcome contributions from any lab with relevant data. Labs with data contributions are invited to be authors in a dataset release paper (like this example from CalMS21) when the data has been compiled and open-sourced.
If you are interested in contributing, please use this Google form to indicate your interest: https://forms.gle/ZuWeL4WcPXorS6Q1A by end of January 2023. We will follow up with you via email.
We also invite you to join our Computational Behavior Slack here.
If you have questions, please contact us via Slack or at mabe.workshop@gmail.com.
Data Specifications
Videos of freely interacting adult mice (either/both sexes), filmed from above, with manual frame-by-frame annotations of one or more social behavior.
Dataset (with annotations) is not publicly available anywhere else online.
Dataset consists of at least 30 around ten-minute videos from an experimental setting.
You acknowledge that we will publicly release pose estimates and manual annotations from this dataset during the competition, and pose, annotation, and videos will later be made publicly available in an online repository (like this) and documented in a dataset release paper (like this), on which you are invited to be an author.
Optional: Ideally the videos have been tracked (ex: with DLC or SLEAP), but if not, we can cover this part.
If you are interested and your dataset is a good fit, contact us by the end of January using Google forms: https://forms.gle/ZuWeL4WcPXorS6Q1A
How your data is used
The compiled dataset across labs will be part of a new machine learning competition aimed at creating mouse social behavior classifiers that generalize: that can detect the behavior across various experimental settings. This dataset will be open-sourced for the community after the competition ends. Our competitions in previous years have had 300+ participants across the globe each year (year 1 and year 2). We are also scaling up the competition this year! We are relocating to Kaggle (run by Google) and will have a larger prize pool totaling $50,000 for winning entries. For the timeline, we aim to launch the challenge in late spring/summer 2023 and run for ~2 months; we're also planning a workshop on computational methods for behavior analysis in summer 2023.
Your data will be crucial for developing new machine learning approaches that can better generalize for behavior analysis, both as a key benchmark for machine learning and also to have better automated analysis approaches for studying behavior.