Sclera biometrics have gained significant popularity among emerging ocular traits in the last few years. In order to evaluate the potential of this trait, a considerable amount of research has been presented in the literature, both employing the sclera individually and in combination with the iris. In spite of those initiatives, sclera biometrics need to be studied more extensively to ascertain their usefulness. Moreover, the sclera segmentation task still requires a significant amount of attention due to challenges associated with the performance of existing techniques when sclera recognition is performed in cross-sensor and cross-resolution scenarios. To investigate these challenges, the 8th Sclera Segmentation and Recognition Benchmarking Competition (SSRBC 2023) will be held in the scope of the 2023 IEEE International Joint Conference on Biometrics (IJCB 2023). SSRBC 2023 will be the 8th in the series of sclera (segmentation and recognition) benchmarking competitions following SSBC 2015, SSRBC 2016, SSERBC 2017, SSBC 2018, SSBC 2019 and SSBC 2020 held in conjunction with BTAS 2015, ICB 2016, IJCB 2017, ICB 2018, 19 and 20, respectively. This competition is co-organized by Technology Innovation Hub (TIH), Indian Statistical Institute Kolkata.

All together 31 research groups registered for SSRBC 2023, out of which 5 took part in the final round and submitted a total of 6 segmentation and 1 recognition models for scoring.

The top-3 winning solutions of SSBC 2020 are:



For more details refer to: https://codalab.lisn.upsaclay.fr/competitions/12050



How to participate?

Registration for the competition can be done by email. If you would like to register and receive the training dataset, please send an email to abhijit.das@hyderabad.bits-pilani.ac.in with the subject line as "SSRBC 2023 registration" with the following information:

Name, Affiliation, Email, Phone number, CV , Mailing Address and signed version of the following form .


Organizers :

Dr. Abhijit Das, BITS Pilani, Hyderabad, India  (abhijit.das@hyderabad.bits-pilani.ac.in)

Dr. Aritra Mukherjee, BITS Pilani, , Hyderabad, India  (a.mukherjee@hyderabad.bits-pilani.ac.in)

Prof. Umapada Pal,  TIH, Indian Statistical Institute, Kolkata, India (umapada@isical.ac.in )

Prof. Peter Peer, University of Ljubljana, Ljubljana, Slovenija (peter.peer @fri.uni-lj.si)

Prof. Vitomir Štruc , University of Ljubljana, Ljubljana, Slovenija (vitomir.struc @fe.uni-lj.si)


Execution

Description of the dataset(s) used for the competition and the available annotations

The competition aims to benchmark the sclera segmentation and recognition tasks with a dataset containing both low and high-resolution images. Three different datasets will be employed for the competition, where two were acquired with a DSLR camera and one by a mobile camera. 

The first dataset, i.e, the multi-angle sclera dataset (MASD), consists of 2624 RGB images taken from 82 identities. Images were collected from both the eyes of each individual, so there are 164 different eyes in total in the dataset. For each individual image, four gaze directions (looking straight, left, right and up) were captured and for each direction 4 images were taken. The subjects from the database are both male and female and with different eye colors, few of them are wearing contact lenses and images were taken at different times of the day. The database contains images with blinking eyes, closed eyes and blurred eyes. High-resolution images stored in JPEG format are provided in the database (7500 x 5000 dimensions). A NIKON D 800 camera and 28300 lenses were used for image capturing. A ground truth or manual sclera segmentation of this dataset is also available. For development purposes, a subset of the database, both eye images and ground truth (1 image for each angle/gaze of the first 30 subjects, i.e. 120 images in total) will be provided to the participants. 

The second dataset MOBIUS) dataset comprises 16717 RGB images of 200 eyes from 100 subjects, cropped and rescaled to 3000 x 1700 pixels. A subset of 3542 images from 35 subjects (70 eyes) is designated for segmentation research and contains high-quality manually generated (and later cleaned with a semi-automatic correction procedure annotations of the sclera, iris, and pupil regions. The dataset again contains 4 gaze directions for each eye, but exhibits a significantly higher degree of variability than other datasets due to the use of 3 different mobile phone cameras (Sony Xperia Z5 Compact, Apple iPhone 6s, and Xiaomi Pocophone F1) for image capture, and 3 ambient settings (i.e., sunny outside; inside with good illumination; and inside with poor illumination). Additionally, data about the subjects (e.g., identity, gender, eye color, age, eyewear, eye conditions and allergies) is also available to facilitate research into various data characteristics and their impact on segmentation performance. 

The third dataset, SBVPI, consists of 1858 RGB images of 110 eyes (i.e., 55 subjects) captured with a DSLR camera (specifically, a Canon EOS 60D with macro lenses). All images were manually cropped to extract the desired ROI while maintaining their aspect ratio, then rescaled to 3000 × 1700 pixels to maintain a consistent image size across the entire dataset. Images in the dataset were captured at the highest resolution and quality settings available in the camera and in a laboratory environment. The dataset contains images taken under 4 different gaze directions, with a minimum of 4 images per direction for each subject. The appearance variability in SBVPI is due to identity, eye color, gender, and age. Manually generated markups of the sclera and periocular regions are present for all images. SBVPI is publicly available for research purposes.

Details on the experimental protocol and result generation/submission procedure,

The competition will address two problems of relevance to IJCB 2023, sclera segmentation and recognition, and will be organized around three tasks:

Evaluation measures: 

● Segmentation task: The evaluation measures will be precision and recall (recall will consider the prior measure for ranking the algorithms). The ground truth of the manually segmented sclera region in an eye image is constructed, which will be used as a baseline.

● Recognition task: For the recognition task, we will consider verification experiments and report the Area Under the ROC Curve (AUC) as our main competition metric. For the summary paper, other relevant performance indicators will also be reported.


Summary paper

All participants, whose submission will yield competitive performance (for segmentation task  F1 >0.3 and for recognition task F1 >0.2) in one of the three tasks, will be invited to be co-authors on the summary paper of the competition. The summary paper will be submitted to IEEE IJCB 2023 and if accepted will appear in IEEE Xplore.


A detailed timeline for the competition:

● Site opens 14th Feb 2023

● Registration starts 14th Feb 2023

● Test dataset available 28th Feb 2023

● Registration closes 10th May 2023

● Algorithm submission deadline 30th May 2023

● Results and report announcement 10th June 2023


Relevant publications

● M. Vitek, A.Das et al., "Exploring Bias in Sclera Segmentation Models: A Group Evaluation Approach," in IEEE Transactions on Information Forensics and Security, vol. 18, pp. 190-205, 2023, doi: 10.1109/TIFS.2022.3216468.

● V. Matej, A. Das et al. , SSBC 2020: Sclera Segmentation Benchmarking Competition in the Mobile Environment, IJCB 2020.

● A. Das, U Pal, M. Blumenstein, C. Wang, Y. He, Y. Zhu, Z. Sun, Sclera Segmentation Benchmarking Competition in Cross-resolution Environment, ICB 2019.