VDU 2023 @ CVPR 2023

The 2nd Workshop on Vision Datasets Understanding

Announcements

VDU 2023 Workshop Videos are now available here.
VDU 2023 Awards:
- Best paper: Consistency and Accuracy of CelebA Attribute Values,
- First Place Award in the 1st DataCV Challenge: YuhaoChen, Shen Zhang,RenjieSong, MegviiResearch Nanjing
- Second Place Award in the 1st DataCV Challenge: ShuyuMiao, Lin Zheng, Hong Jin, Ant Group
Paper submission deadline is the 20th of March 2023. More information see here.
The first DataCV Challenge will be held with this workshop. Training data has been released on 30th January. More details see here.
VDU 2023 will be held in conjunction with CVPR 2023 in Vancouver. More details to follow.
Information about last year's workshop, VDU 2022, is available here.

Overview

Data is the fuel of computer vision, on which state-of-the-art systems are built. A robust object detection system not only needs a strong model architecture and learning algorithms but also relies on a comprehensive large-scale training set. Despite the pivotal significance f datasets, existing research in computer vision is usually algorithm centric. Comparing the number of algorithm-centric works in domain adaptation, the quantitative understanding of the domain gap is much more limited. As a result, there are currently few investigations into the representations of datasets, while in contrast, an abundance of literature concerns ways to represent images or videos, essential elements in datasets.

The 2nd VDU workshop aims to bring together research works and discussions focusing on analyzing vision datasets, as opposed to the commonly seen algorithm-centric counterparts. Specifically, the following topics are of interest in this workshop.

Properties and attributes of vision datasets
Application of dataset-level analysis
Representations of and similarities between vision datasets
Improving vision dataset quality through generation and simulation.
Evaluating model accuracy under various test environments.

In summary, the questions related to this workshop include but are not limited to:

Can vision datasets be analyzed on a large scale?
How to holistically understand the visual semantics contained in a dataset?
How to define vision-related properties and problems on the dataset level?
How can we improve algorithm design by better understanding vision datasets?
Can we predict the performance of an existing model in a new dataset?
What are good dataset representations? Can they be hand-crafted, learned through neural nets or a combination of both?
How do we measure similarities between datasets?
How to measure dataset bias and fairness?
Can we improve training data quality through data engineering or simulation?
How to efficiently create labelled datasets under new environments?
How to create realistic datasets that serve our real-world application purpose?
How can we alleviate the need for large-scale labelled datasets in deep learning?

DataCV Challenge

In 2023, the DataCV Challenge aims to estimate the difficulty of various test sets which do not have ground truths, which is also known as unsupervised model evaluation or label-free model evaluation. This problem is potentially very useful to detect model failure when deployed in certain environments.

Challenge website: https://sites.google.com/view/vdu-cvpr23/competition

Important Dates for the Workshop

Monday, March 20th , 2023: Workshop paper submission closed
Friday, March 31st , 2023: Author notification of paper acceptance
Thursday, April 6th , 2023: Camera ready (may be amended as per the camera-ready deadline of the main conference)

Note: The above deadlines apply to those who want to have their papers included in the proceedings. If you prefer not to be included into the proceedings but still want to share your work with the community, please contact the organizing committee to find out possible solutions.

Important Dates for DataCV Challenge

Monday, January 23rd , 2023: Training and validation data release
Thursday, March 9th , 2023: Test data release
Monday, March 13th , 2023: Result submission closed
Monday, March 20th , 2023: Workshop paper submission closed
Friday, March 31st , 2023: Author notification of paper acceptance
Thursday, April 6th , 2023: Camera ready (may be amended as per the camera-ready deadline of the main conference)

Note: If your team would like to compete for an award (1st place, 2nd place etc), it is mandatory to submit a paper to the workshop and release the code publicly. More details are provided on the DataCV Challenge website

Paper Submission

To ensure the high quality of the accepted papers, all submissions will be evaluated by research and industry experts from the corresponding fields. Reviewing will be double-blind and we will accept submissions on work that is not published, currently under review, and already published. All accepted workshop papers will be published in the CVPR 2023 Workshop Proceedings by Computer Vision Foundation Open Access. The authors of all accepted papers (oral/spotlight/posters) will be invited to present their work during the actual workshop event at CVPR 2023.

Paper submission has to be in English, in pdf format, and at most 8 pages (excluding references) in double column. The paper format must follow the same guidelines as for all CVPR 2023 submissions. The author kit provides a LaTeX2e template for paper submissions. Please refer to this kit for detailed formatting instructions.

Submission Website: https://cmt3.research.microsoft.com/VDUCVPR2023

For information about whether the workshop will be in-person, virtual, or hybrid please visit the CVPR 2023 website.

Program

Location: TBD

Schedule: June 18th 1:30 pm – 5:30 pm

1:30 pm – 1:40 pm Workshop Kick off and Opening Comments

1:40 pm – 2:10 pm First Keynote Speech (25 mins for talk and 5mins for Q&A)

2:10 pm – 3:10 pm 4 Oral Presentations (13 mins for talk and 2 mins for Q&A)

2:10 pm – 2:25 pm ID-1: Consistency and Accuracy of CelebA Attribute Values
2:25 pm – 2:40 pm ID-3: Compensation Learning in Semantic Segmentation
2:40 pm – 2:55 pm ID-5: Digital Twin Tracking Dataset (DTTD): A New RGB+Depth 3D Dataset for Longer-Range Object Tracking Applications
2:55 pm – 3:10 pm ID-6: Assessing Domain Generalization of Semantic Segmenters with Synthetic Data

3:10 pm – 3:25 pm Coffee Break

3:25 pm – 3:55 pm Second Keynote Speech (25 mins for talk and 5mins for Q&A)

3:55 pm – 5:00 pm 4 Oral Presentations (13 mins for talk and 2 mins for Q&A)

3:55 pm – 4:00 pm Challenge Introduction and Winner Announcement
4:00 pm – 4:15 pm ID-4: Scoring Your Prediction on Unseen Data
4:15 pm – 4:30 pm ID-7: K-means Clustering Based Feature Consistency Alignment for Label-free Model Evaluation
4:30 pm – 4:45 pm ID-10: Exploring Video Frame Redundancies for Efficient Data Sampling and Annotation in Instance Segmentation
4:45 pm – 5:00 pm ID-11: WEDGE: A Multi-Weather Autonomous Driving Dataset Built from Generative Vision-Language Models

5:00 pm -5:30 pm Poster Session