9th Sclera Segmentation and Benchmarking Competition (SSBC 2025)
Theme: Privacy-aware model learning through synthetic data
Theme: Privacy-aware model learning through synthetic data
The Sclera Segmentation and Benchmarking Competition (SSBC 2025) aims to advance research in sclera biometrics by focusing on segmentation models trained (exclusively) on synthetic ocular data. Despite significant progress in ocular biometrics, sclera segmentation remains a challenging task due to the limited availability of large-scale datasets and privacy concerns associated with real-world data collection.
SSBC 2025 will explore the potential of synthetic images and weakly labeled segmentation masks to train robust machine learning models capable of handling real-world conditions. Participants will benchmark the performance of their segmentation models using synthetic ocular images and automatically generated (weak) ground truth masks. The goal of SSBC 2025 is to push the boundaries of current machine learning techniques in ocular biometrics, particularly in scenarios where manually annotated datasets are scarce. By focusing on synthetic data, the competition seeks to drive innovation in developing more generalized and effective segmentation models, ultimately reducing reliance on costly and time-consuming manual annotations.
Registration for the competition can be done via email. If you would like to register and receive the training dataset and starter kit, please send an email to abhijit.das@hyderabad.bits-pilani.ac.in and matej.vitek@fri.uni-lj.si with the subject line as "SSBC 2025 registration" with the following information:
Name, Affiliation, Email, Phone number, CV , and Mailing Address.
Abhijit Das, BITS Pilani
Umapada Pal, Indian Statistical Institute, Kolkata
Peter Peer, University of Ljubljana
Vitomir Štruc, University of Ljubljana
Darian Tomašević, University of Ljubljana
Matej Vitek, University of Ljubljana
In this competition, we aim to benchmark the performance of sclera segmentation models trained with synthetic ocular images and corresponding automatically generated (weak) ground truth segmentation masks. The primary motivation behind this challenge is to push the boundaries of current machine learning techniques in handling ocular biometrics, particularly under conditions where traditional, manually annotated datasets are scarce or unavailable. By focusing on synthetic images and weakly labelled data, we hope to stimulate research into more robust and generalized machine learning models that can perform accurately even when faced with imperfect or limited training data. This approach not only has the potential to significantly reduce the time and cost associated with manual annotation but also paves the way for advancements in areas where high-quality, real-world data is hard to come by. Through this competition, we aim to foster a collaborative effort within the research community to address these challenges, ultimately contributing to more effective and accessible biometric identification technologies.
The competition will address the problem of sclera segmentation, where participants will have to learn their models on: (1) synthetic data only and (2) a combination of real and synthetic data. For the evaluation, participants will have to provide results on:
Segmentation with synthetic data: for this task, participants will have to learn segmentation models on two sets of training data: (1) synthetic training data only, and (2) a combination of real and synthetic data, and then test the learned models on a synthetic test dataset. Here, the synthetic test data will consist of images from two data generators (one producing Caucasian and the other Asian ocular images). Segmentation masks generated on the test data will have to be submitted for scoring. Additionally, participant will also have to submit complete models and training scripts, so the organizers can validate the results.
Segmentation with real data: for this task, participants will have to learn segmentation models on two sets of training data: (1) synthetic training data only, and (2) a combination of real and synthetic data, and then test the learned models on the real-world datasets. In this case, manually generated (ground truth) segmentation masks will be used for generating the performance scores.
All participants, whose submission will yield competitive performance (an F1>0.7 for one of the segmentation tasks), will be invited to be co-authors on the summary paper of the competition. The summary paper will be submitted to IEEE IJCB 2025 and if accepted will appear in IEEE Xplore.
A detailed document with instructions on the competition is available HERE.
Site opens: 4th March 2025
Registration starts: 4th March 2025
Starter kit available: 24th March 2025
Training data available: 24th March 2025
Test data available: 26th May 2025
Result submission: 2nd June 2025
Registration closes: 26th May 2025
Summary paper ready: 22nd June 2025
The competition was performed in two tracks: (i) the Synthetic track, featuring models trained on synthetic data only and (ii) the Mixed track with models trained on both synthetic and real-world data. The approaches were ranked according to the harmonic mean F1-score across the three evaluation datasets.
The winner of the Synthetic track was the Couger Inc. team with their submission, SwinDANet. The Mixed track had two joint winners, whose F1-scores were within 0.001 of each other: SAM-Iris (from the Idiap – HES-SO team) and ShapeGAN-DLV3+ (submitted by the Ahmedabad University team). The full rankings of both tracks are included below, while a more detailed analysis of the results will be available in the summary paper submitted to the IJCB conference.
a) Synthetic
b) Mixed
M. Vitek, A.Das et al., "Exploring Bias in Sclera Segmentation Models: A Group Evaluation Approach," in IEEE Transactions on Information Forensics and Security, vol. 18, pp. 190-205, 2023. [PDF]
D. Tomašević; P. Peer, V. Štruc, BiOcularGAN: Bimodal Synthesis and Annotation of Ocular Images, IJCB 2022, pp. 1-10, 2022. [PDF]
M. Vitek, A. Das et al. , SSBC 2020: Sclera Segmentation Benchmarking Competition in the Mobile Environment, IJCB 2020. [PDF]
A. Das, U Pal, M. Blumenstein, C. Wang, Y. He, Y. Zhu, Z. Sun, Sclera Segmentation Benchmarking Competition in Cross-resolution Environment, ICB 2019. [PDF]