Challenge

The San Vitale challenge

This competition is divided into two distinct tracks, each divided into two sub-tracks. In all tracks, the goal is to successfully (partially) reconstruct disks of increasing levels of difficulty. 

Tracks

The tracks are divided as follows:

• Track 1: for each disk, all the given fragments are used in the reconstruction.

  • Sub-track 1.1: participants can use the given fragments’ polylines.

  • Sub-track 1.2: participants must employ only the raw images, without using the given polylines.

• Track 2: for each disk, only some fragments are used in the reconstruction; some fragments are outliers and are not needed.

  • Sub-track 2.1: participants can use the given fragments’ polylines.

  • Sub-track 2.2: participants must employ only the raw images, without using the given polylines.

Registration

Please, register your team here: https://forms.office.com/e/PMivuACXn1

Paper

The best teams of each sub-track will be invited to contribute as co-authors in the summary paper of the San Vitale Challenge.

This paper will be published in the proceedings of the ECCV 2024 conference.

Data

To download the files, after the registration, please visit the challenge Codalab site (https://codalab.lisn.upsaclay.fr/competitions/19244).

The dataset is organized in 2 folders:

• Train: the folder that can be used for learning procedures and algorithm development. Here, there are also ground truth annotations.

• Test: the folder on which you have to run your solution and the metrics are computed.

In each folder, you will find:

• 'gt.png': the image of the whole or partial disc manually reconstructed (only in the train folder).

• 'adjacency.txt': the adjacency matrix of the fragments.

• 'xxx.png': images of the glass fragments in full resolution ('xxx' may be letters or numbers).

• 'xxx.txt': glass fragment polyline (for sub-tracks 1.1 and 2.1, 'xxx' may be letters or numbers).

• 'xxx_pl.png': images with the polyline ('xxx' may be letters or numbers).

For Track 2, the outliers are not listed in the adjacency matrix.

Each polyline is provided in a .txt file with the following format:

• x \t y coordinates

• Points are manually annotated clockwise. 

• The first and the last points of the polyline are the same.

Example of the annotated polyline (in green):

Results Submission

• Results must be submitted to CodaLab: https://codalab.lisn.upsaclay.fr/competitions/19244

• Each line of the adjacency matrix must start with the reference of the fragment (number or letter), followed by a colon and whitespace, and the list of connected fragments separated by a whitespace.
  Example:

  542: 345 234 456
345: 234 542
37:
...

• Note that it is possible to add a piece without connections but part of the final disk (e.g. piece 37).

• Track 2 contains at least one outlier fragment in each folder; outliers in Track 2 should not appear in the adjacency matrix.

• The adjacency list must be symmetrical (i.e. if fragment A is connected to fragment B, then fragment B must also be connected to fragment A).

• The output of the proposed algorithm must be an adjacency list saved in a trackXY/z.txt file, where XY is the chosen track, and z is number of the fragment.

• Missing trackXY folders will mean that the team is not participating in those tracks.

• The starting kit contains an empty submission that must be filled with the adjacency lists.

Scoring

Submissions will be judged using a scoring system:

• A fragment is present in both the submission and ground truth: +1 point

• A fragment is present in the submission but not in the ground truth: -0.5 points

• A fragment is present in the ground truth but not in the submission: -1 point

• A connection between two fragments is present in both the submission and ground truth: +2 points

• A connection between two fragments is present in the submission but not in the ground truth: -0.25 points

• A connection between two fragments is present in the ground truth but not in the submission: -0.5 points

The score for each disk is then normalized between 0 and 1. Out of bounds score are clipped to this interval. The lower bound score is determined by computing the average score of 1000 solutions, randomly selecting fragments and connections.

Rules of the Challenge 

• There are no limits on the maximum number of submissions; however, only the last will count towards the final ranking.

• Fragments of different folders must not be used when reconstructing a particular disk. 

• Submissions must be fully reproducible: the full code used to generate the proposed solution for each track must be included in the final submission package, along with clear instructions on how to run it. Note that it is not required to submit the code in all other submission packages.

• The proposed algorithms must be fully automatic and output a disk reconstruction without any human intervention, starting from the allowed inputs.

• Submissions that are not reproducible or do not follow the required structure are disqualified and will not be considered for the final evaluation.

Rules of Teams

• A team is composed of at most 10 participants.

• The team’s participants may change during the whole submission period; after that, no more substitutions can take place.

• Teams may participate in one or many sub-tracks.