(last update of this page: May 2024)
Participation in Evaluations of Technology
Leader of the participation in international evaluations of technology using developments of own research, with submissions usually ranked in the first two positions (mostly among ten or more participants). A research problem is usually defined, including an evaluation protocol. Training data is made available to the participants to tune their algorithms. Participants then submit their algorithms, which are tested by the organizers with sequestered data (i.e. not seen previously by the participants).
In several cases, the work is based on research done by supervised MSc or PhD students. The algorithms submitted are in most cases described in publications. Sometimes there is also a joint publication by organizers and participants describing the competition and results.
EFaR 2023: Efficient Face Recognition Competition
Organizer: Fraunhofer IGD, Germany and TU Darmstadt, Germany, in conjunction with the International Joint Conference on Biometrics (IJCB-2023)
Objective: efficient and lightweight face recognition
Role: system development, integration and evaluation, applying own research work
Results: 7th/9 (2-5MB models), 8th/8 (<2MB models)
Joint publication by organizers and participants describing the competition:
J. N. Kolf, F. Boutros, J. Elliesen, M. Theuerkauf, N. Damer, M. Alansarir, O. A. Hay, S. Alansari, S. Javed, N. Werghi, K. Grm, V. Struc, F. Alonso-Fernandez, K. Hernandez Diaz, J. Bigun, A. George, C. Ecabert, H. O. Shahreza, K. Kotwal, S. Marcel, I. Medvedev, J. Bo, D. Nunes, A. Hassanpour, P. Khatiwada, A. A. Toor, B. Yang, “EFaR 2023: Efficient Face Recognition Competition”, Proc. IEEE/IAPR International Joint Conference on Biometrics, IJCB, Ljubljana, Slovenia, Sep 25-28, 2023 https://arxiv.org/abs/2308.04168
Paper describing the basis of training the submitted methods:
F. Alonso-Fernandez, J. Barrachina, K. Hernandez Diaz, J. Bigun, “SqueezeFacePoseNet: Lightweight Face Verification Across Different Poses for Mobile Platforms”, Proc. IAPR TC4 Workshop on Mobile and Wearable Biometrics, WMWB, in conjunction with IEEE/IAPR Intl Conf on Pattern Recognition, ICPR, 2020, Milan, Italy, 10-15 January 2021 (online) (Oral) https://doi.org/10.1007/978-3-030-68793-9_10, https://arxiv.org/abs/2007.08566, https://youtu.be/suJmO8IWp8k (collaboration due to visit to the company FacePhi Biometria in Spain)
DFGC 2022: The Second DeepFake Game Competition
Organizer: Chinese Academy of Sciences, in conjunction with the International Joint Conference on Biometrics (IJCB-2022)
Objective: creation and detection of deepfakes
Role: work of co-supervised PhD student Felix Rosberg
Results: 3rd/35 in the deepfake creation task
Paper of the competition:
B. Peng, W. Xiang, W. Wang, J. Dong, Z. Sun, Z. Lei, S. Lyu, "DFGC 2022: The Second DeepFake Game Competition," 2022 IEEE International Joint Conference on Biometrics (IJCB), Abu Dhabi, United Arab Emirates, 2022, pp. 1-10, doi: 10.1109/IJCB54206.2022.10007991, https://ieeexplore.ieee.org/document/10007991
Paper describing the submitted method:
F. Rosberg, E. Aksoy, F. Alonso-Fernandez, C. Englund, “FaceTrans: Pose- and Occlusion-Aware High Fidelity Face Swapping”, Proc. IEEE/CVF Winter Conference on Applications of Computer Vision, WACV, Waikoloa, Hawaii, Jan 3-7, 2023 http://arxiv.org/abs/2210.10473
CROSS-EYED 2017: 2nd Cross-Spectrum Iris/Periocular Recognition
Organizer: University of Reading (UK), in conjunction with the International Joint Conference on Biometrics (IJCB-2017)
Objective: performance assessment of periocular recognition algorithms, with emphasis on matching of data from different spectra (near-infrared vs. visible)
Role: system development, integration and evaluation, applying own postdoctoral research work
Results: system ranked 1st/12 in the final submission
Web: https://sites.google.com/site/crossspectrumcompetition/
Joint publication by organizers and participants describing the competition:
A. F. Sequeira, L. Chen, J. Ferryman, P. Wild, F. Alonso-Fernandez, J. Bigun, T. Freitas Pereira, S. Marcel, V. Kanhangad, K. B. Raja, R. Raghavendra, C. Busch, “Cross-Eyed - Cross-Spectral Iris/Periocular Recognition Database and Competition”, Proc. IEEE/IAPR International Joint Conference on Biometrics, IJCB, Denver, CO, Oct 1-4, 2014
CROSS-EYED 2016: 1st Cross-Spectrum Iris/Periocular Recognition
Organizer: University of Reading (UK), in conjunction with the 8th Conference on Biometrics: Theory, Applications, and Systems (BTAS-2016)
Objective: performance assessment of periocular recognition algorithms, with emphasis on matching of data from different spectra (near-infrared vs. visible)
Role: system development, integration and evaluation, applying own postdoctoral research work
Results: system ranked 1st/5 in intermediate results, 1st/7 in the final submission
Web: https://sites.google.com/site/crossspectrumcompetition/cross-eyed-2016
Joint publication by organizers and participants describing the competition:
A. F. Sequeira, L. Chen, J. Ferryman, F. Alonso-Fernandez, J. Bigun, K. B. Raja, R. Raghavendra, C. Busch, P. Wild, “Cross-Eyed - Cross-Spectral Iris/Periocular Recognition Database and Competition”, Proc. 15th International Conference of the Biometrics Special Interest Group, BIOSIG, Darmstadt, Germany, 21-23 Sept 2016
ICIR2013: First ICB Competition on Iris Recognition
Organizer: Institute of Automation, Chinese Academy of Sciences (CASIA), in conjunction with the 6th International Conference on Biometrics (ICB-2013)
Objective: performance assessment of iris recognition algorithms, with data including motion blur, non-linear deformation, eyeglasses and specular reflections
Role: system development, integration and evaluation, applying own postdoctoral research work
Results: system ranked 2nd/13 (1st position among academic participants)
Algorithms based on developments described in papers:
F. Alonso-Fernandez, J. Bigun, “Periocular Recognition Using Retinotopic Sampling and Gabor Decomposition”, Intl Workshop “What's in a Face?” WIAF, in conjunction with the European Conference on Computer Vision, ECCV, Springer LNCS-7584, pp. 309-318, Firenze, Italy, October 7-13, 2012
F. Alonso-Fernandez, J. Bigun, “Iris Boundaries Segmentation Using the Generalized Structure Tensor. A Study on the Effects on Image Degradation”, Intl Conf on Biometrics: Theory, Apps and Systems, BTAS, Washington DC, September 23-26, 2012
Joint publication by organizers and participants describing the competition:
M. Zhang, J. Liu, Z. Sun, W. Su, F. Alonso-Fernandez, V. Némesin, N. Othman, K. Noda, P. Li, E. Hoyle, A. Joshi, “The First ICB Competition on Iris Recognition”, Proc. IEEE/IAPR International Joint Conference on Biometrics, IJCB, Clearwater (Tampa), FL, 29 Sept – 2 Oct, 2014
4NSigComp2010: Forensic Signature Verification Competition (off-line)
Organizer: The Netherlands Forensic Institute (NFI), in conjunction with the 12th International Conference on Frontiers of Handwriting Recognition (ICFHR)
Objective: performance assessment of signature verification systems on a forensic-like data set of off-line images, including skilled forgeries and disguised signatures
Role: coordinator of the submission (incl. system development, integration and evaluation), applying own PhD research work, developed in conjunction with two supervised MSc students
Results: system ranked 2nd/10
Web: https://www.isical.ac.in/~icfhr2010/CallforParticipation4NSigComp2010.html
Algorithms based on developments described in papers (name of students underlined):
F. Alonso-Fernandez, J. Fierrez, A. Gilperez, J. Ortega-Garcia, “Impact of time variability in off-line writer identification and verification”, Proc. IEEE ISPA, Spec. Sess. on Signal and Image Processing for Biometrics, Salzburg, Austria, 2009-09-16
A. Gilperez, F. Alonso-Fernandez, S. Pecharroman, J. Fierrez, J. Ortega-Garcia, “Off-line signature verification using contour features”, Intl Conf on Frontiers in Handwriting Recognition, ICFHR, Montréal, Québec, August 19-21, 2008
SigComp 2009: Signature Competition, On- and Offline Skilled Forgeries
Organizer: The Netherlands Forensic Institute (NFI), in conjunction with the 10th International Conference on Document Analysis and Recognition (ICDAR)
Objective: performance assessment of signature verification systems on a forensic-like data set
Role: coordinator of the submission (incl. system development, integration and evaluation), applying own PhD research work, developed in conjunction with two supervised MSc students and one PhD Student (Marcos Martinez-Diaz)
Results: system ranked 2nd/15 (online signatures), 5th/8 (offline signatures)
Algorithms based on developments described in papers (name of students underlined):
F. Alonso-Fernandez, J. Fierrez, M. Martinez-Diaz, J. Ortega-Garcia, “Fusion of static image and dynamic information for signature verification”, IEEE Proc. Intl. Conf. on Image Processing, ICIP, El Cairo, Egypt, 2009-11-07
F. Alonso-Fernandez, J. Fierrez, A. Gilperez, J. Ortega-Garcia, “Impact of time variability in off-line writer identification and verification”, Proc. IEEE ISPA, Spec. Sess. on Signal and Image Processing for Biometrics, Salzburg, Austria, 2009-09-16
A. Gilperez, F. Alonso-Fernandez, S. Pecharroman, J. Fierrez, J. Ortega-Garcia, “Off-line signature verification using contour features”, Intl Conf on Frontiers in Handwriting Recognition, ICFHR, Montréal, Québec, August 19-21, 2008
Competition described in the paper:
V. L. Blankers, C. E. van den Heuvel, K. Y. Franke, L. G. Vuurpijl. ”The ICDAR 2009 signature verification competition”. Proc. 10th Intl. Conf. on Document Analysis and Recognition, ICDAR, pp 1403–1407, 2009
LivDet 2009: Fingerprint Liveness Detection Competition
Organizer: University of Cagliari (Italy) and Clarkson University (USA), in conjunction with the 5th International Conference on Image Analysis and Processing (ICIAP)
Objective: to compare different methodologies for software-based fingerprint liveness detection
Role: system development, integration and evaluation applying own PhD research work, in conjunction with a PhD Student (Javier Galbally)
Results: system ranked 2nd/4
Algorithm described in the papers (student name underlined):
J. Galbally, F. Alonso-Fernandez, J. Fierrez, J. Ortega-Garcia, “A High Performance Fingerprint Liveness Detection Method Based on Quality Related Features”, Elsevier Future Generation Computer Systems Journal, 28 (1): 311-321, January 2012, ISSN 0167-739X
J. Galbally, F. Alonso-Fernandez, J. Fierrez, J. Ortega-Garcia, “Fingerprint Liveness Detection Based on Quality Measures”, IEEE Proc. Intl. Conf. on Biometrics, Identity and Security, BIDS, Tampa, FL, Sept 22-23, 2009
BSEC 2009: The BioSecure Signature Evaluation Campaign
Organizer: TELECOM & Management SudParis, in conjunction with the International Conference on Biometrics 2009 (ICB'09)
Objective: to compare signature recognition systems in mobility acquisition conditions
Role: system development, integration and evaluation using own technology of the research group, in conjunction with a PhD Student (Marcos Martinez-Diaz)
Results: system ranked 2nd/1st of 14 (2 scenarios)
Algorithm described in the papers (student name underlined):
F. Alonso-Fernandez, J. Fierrez, M. Martinez-Diaz, J. Ortega-Garcia, “Fusion of static image and dynamic information for signature verification”, IEEE Proc. Intl. Conf. on Image Processing, ICIP, El Cairo, Egypt, 2009-11-07
BMEC 2007: The Biosecure Multimodal Evaluation Campaign
Organizer: BioSecure Network of Excellence
Objective: to compare fusion algorithms under several conditions (cross-device matching, mobility conditions)
Role: coordinator of the submission in the cross-device scenario (submission included as research work in own PhD Thesis), support of a co-worker (Daniel Ramos) in the mobility scenario (incl. system development, integration and evaluation)
Results: system ranked 2nd/1st of 14/9 (cross-device, mobility scenarios respectively)
Algorithms described in the papers:
F. Alonso-Fernandez, J. Fierrez, D. Ramos, J. Gonzalez-Rodriguez, “Quality-Based Conditional Processing in Multi-Biometrics: Application to Sensor Interoperability”, IEEE Trans. On Systems, Man & Cybernetics, part A, 40 (6): 1168-1179, November 2010, ISSN 1083-4427
F. Alonso-Fernandez, J. Fierrez, D. Ramos and J. Ortega-Garcia, "Dealing with sensor interoperability in multi-biometrics: the UPM experience at the BioSecure Multimodal Evaluation 2007", in Proc. Defense and Security Symposium, Biometric Technologies for Human Identification, BTHI, Proc. SPIE, Orlando, USA, March 2008
Joint publication by organizers and participants describing the competition:
N. Poh, T. Bourlai, J. Kittler, L. Allano, F. Alonso-Fernandez, O. Ambekar, J. Baker, B. Dorizzi, O. Fatukasi, J. Fierrez, H. Ganster, J. Ortega-Garcia, D. Maurer, A. A. Salah, T. Scheidat and C. Vielhauer, "Benchmarking Quality-dependent and Cost-sensitive Score-level Multimodal Biometric Fusion Algorithms", IEEE Trans. on Information Forensics and Security, 4 (4): 849-866, Dec. 2009, ISSN: 1556-6013