The aim of the Keystroke Verification Challenge - onGoing (KVC-onGoing) is providing a public and reproducible way to benchmark keystroke-based user recognition systems in desktop and mobile scenarios, using large-scale databases and a standard experimental protocol.

The KVC-onGoing is based on one of the limited-time challenge held within the 2023 IEEE International Conference on Big Data (IEEE BigData 2023), December 15-18, Sorrento, Italy. 

The KVC-onGoing considers two different tasks based on popular and challenging scenarios:

• Desktop task

• Mobile task

KVC-onGoing in brief: what does it offer and what is new?

1) The size of the databases, suitable for deep learning applications:

• Desktop task:

Development set: 115,120 subjects

Evaluation set: 15,000 subjects

• Mobile task:

Development set: 40,639 subjects

Evaluation set: 5,000 subjects

In both scenarios, the data in the development set consist in several sessions per subject, each of them containing a sentence of transcript text (variable content, but not fully free-text). Read more. The demographic labels of the development sets are included with the data.

2) Easy participation: to simplify the process of approaching the challenge, we provide some Python scripts to start off, that include respectively the training of a baseline RNN model and the creation of the file with the evaluation scores, ready for the submission on Codalab, without the need to disclose your system. Read more.

3) Submission: you can easily benchmark your system in the Codalab platform by submitting your evaluation scores. 

A thorough presentation of all aspects and first benchmark of the challenge is available:

• Giuseppe Stragapede, Ruben Vera-Rodriguez, Ruben Tolosana, Aythami Morales, Naser Damer, Julian Fierrez, Javier Ortega-Garcia (2023). Keystroke Verification Challenge (KVC): Biometric and Fairness Benchmark Evaluation. IEEE Access.  (arXiv)

• Giuseppe Stragapede, Ruben Vera-Rodriguez, Ruben Tolosana, Aythami Morales, Ivan DeAndres-Tame, Naser Damer, Julian Fierrez, Javier-Ortega Garcia, Nahuel Gonzalez, Andrei Shadrikov, Dmitrii Gordin, Leon Schmitt, Daniel Wimmer, Christoph Großmann, Joerdis Krieger, Florian Heinz, Ron Krestel, Christoffer Mayer, Simon Haberl, Helena Gschrey, Yosuke Yamagishi, Sanjay Saha, Sanka Rasnayaka, Sandareka Wickramanayake, Terence Sim, Weronika Gutfeter, Adam Baran, Mateusz Krzysztón, Przemysław Jaskół (2023). IEEE BigData 2023 Keystroke Verification Challenge (KVC). Proc. of the IEEE Int. Conf. on Big Data. (arXiv)

Main Results (of the limited-time KVC organized within IEEE BigData 2023, December 15-18, Sorrento, Italy)

DESKTOP TASK

Position Team Global EER↓ (%) FNMR↓ @10% FMR (%) FNMR↓ @1% FMR (%) FNMR↓ @0.1% FMR (%)

1 LSIA 3.33 0.51 11.96 44.17

2 VeriKVC 4.03 1.05 18.79 59.05

3 Keystroke Wizards 5.22 1.62 27.98 67.86

4 U-CRISPER 6.19 2.68 35.24 74.77

5 YYama 6.41 2.88 36.96 74.16

6 Challenger∗ 6.79 3.52 39.36 76.93

7 BioSense 10.85 12.0 54.59 84.94

MOBILE TASK

Position Team Global EER↓ (%) FNMR↓ @10% FMR (%) FNMR↓ @1% FMR (%) FNMR↓ @0.1% FMR (%)

1 LSIA 3.61 0.6 17.44 63.62

2 VeriKVC 3.78 0.95 18.39 65.88

3 Keystroke Wizards 5.83 1.93 41.58 84.14

4 U-CRISPER 8.76 6.68 67.15 94.79

5 YYama 4.16 0.72 24.41 69.62

6 Challenger∗ 5.19 1.55 32.89 77.46

7 BioSense 11.83 14.43 60.48 88.11

∗Participated in the Unrestricted task.

We encourage the participation of as many research and industry groups as possible, so that this challenge will provide a complete panorama of the state of the art in the field of user recognition based on keystroke dynamics. 

Please check the guide to participation pages!

The KVC-onGoing is organized by the BiDA-Lab (website) at the Universidad Autónoma de Madrid.

Codalab: https://codalab.lisn.upsaclay.fr/competitions/14063

GitHub: https://github.com/giuseppe-stragapede/Keystroke-Verification-Challenge

Contact: g.stragapede0@gmail.com

NEWS

12/12/2023: The ongoing challenge has started!

19/06/2023: A new CodaLab page is created (link) in order to provide a dedicated track to off-the-shelf commercial systems (but open to anyone). See the Important Update section here for more information.

29/05/2023: This website was created!