The aim of the Keystroke Verification Challenge - onGoing (KVC-onGoing) is providing a public and reproducible way to benchmark keystroke-based user recognition systems in desktop and mobile scenarios, using large-scale databases and a standard experimental protocol.

The KVC-onGoing is based on one of the limited-time challenge held within the 2023 IEEE International Conference on Big Data (IEEE BigData 2023), December 15-18, Sorrento, Italy. 

The KVC-onGoing considers two different tasks based on popular and challenging scenarios:

• Desktop task

• Mobile task

KVC-onGoing in brief: what does it offer and what is new?

1) The size of the databases, suitable for deep learning applications:

• Desktop task:

Development set: 115,120 subjects

Evaluation set: 15,000 subjects

• Mobile task:

Development set: 40,639 subjects

Evaluation set: 5,000 subjects

In both scenarios, the data in the development set consist in several sessions per subject, each of them containing a sentence of transcript text (variable content, but not fully free-text). Read more. The demographic labels of the development sets are included with the data.

2) Easy participation: to simplify the process of approaching the challenge, we provide two Python scripts to start off, that include respectively the training of a baseline RNN model and the creation of the file with the evaluation scores, ready for the submission on Codalab, without the need to disclose your system. Read more.

3) Submission: you can easily benchmark your system in the Codalab platform by submitting your evaluation scores. 

A thorough presentation of all aspects and first benchmark of the challenge is available:

• Stragapede, G., Vera-Rodriguez, R., Tolosana, R., Morales, A., Damer, N., Fierrez, J., & Ortega-Garcia, J. (2023). Keystroke Verification Challenge (KVC): Biometric and Fairness Benchmark Evaluation. IEEE Access. 

Main Results (of the limited-time KVC organized within IEEE BigData 2023, December 15-18, Sorrento, Italy)

DESKTOP TASK

Position Team Global EER↓ (%) FNMR↓ @10% FMR (%) FNMR↓ @1% FMR (%) FNMR↓ @0.1% FMR (%)

1 LSIA 3.33 0.51 11.96 44.17

2 VeriKVC 4.03 1.05 18.79 59.05

3 Keystroke Wizards 5.22 1.62 27.98 67.86

4 U-CRISPER 6.19 2.68 35.24 74.77

5 YYama 6.41 2.88 36.96 74.16

6 Challenger∗ 6.79 3.52 39.36 76.93

7 BioSense 10.85 12.0 54.59 84.94

MOBILE TASK

Position Team Global EER↓ (%) FNMR↓ @10% FMR (%) FNMR↓ @1% FMR (%) FNMR↓ @0.1% FMR (%)

1 LSIA 3.61 0.6 17.44 63.62

2 VeriKVC 3.78 0.95 18.39 65.88

3 Keystroke Wizards 5.83 1.93 41.58 84.14

4 U-CRISPER 8.76 6.68 67.15 94.79

5 YYama 4.16 0.72 24.41 69.62

6 Challenger∗ 5.19 1.55 32.89 77.46

7 BioSense 11.83 14.43 60.48 88.11

∗Participated in the Unrestricted task.

We encourage the participation of as many research and industry groups as possible, so that this challenge will provide a complete panorama of the state of the art in the field of user recognition based on keystroke dynamics. 

Please check the guide to participation pages!

The KVC-onGoing is organized by the BiDA-Lab at the Universidad Autónoma de Madrid.

Codalab: https://codalab.lisn.upsaclay.fr/competitions/14063

Contact: giuseppe.stragapede@uam.es

NEWS

12/12/2023: The ongoing challenge has started!

19/06/2023: A new CodaLab page is created (link) in order to provide a dedicated track to off-the-shelf commercial systems (but open to anyone). See the Important Update section here for more information.

29/05/2023: This website was created!