Second Intermediate Results

(This information was updated on the 5th April 2017.)

Five teams submitted their methods to the competition: IDIAP, IITIndore, HH, NTNU and TIGER.

Results of the PERIOCULAR TASK
IDIAP, IITIndore, HH, NTNU and TIGER submitted periocular methods.

Table 1. Results of the 13 methods submitted (error rate in %)
 MethodIntermediate  Submission
no.
Rank
 GF2
GFRR@GFAR=0.01
 EER
 NTNU1 210.78
0.86
 IDIAP2 122.08
1.56
 HH1 23 2.301.60
 IDIAP1 14
3.712.38
 NTNU3 154.733.05
 NTNU2 16
 4.80 3.05
 HH2 27
 12.628.52
 HH3 28
 14.18 9.33
  NTNU1_12 9 16.40 7.59
 IITIndore1 2 10 43.9921.53
 IITIndore2 2 11 51.91 30.90
 TIGER 212
 75.39 21.42
 NTNU4 2 1376.37
26.84


Results of the IRIS TASK
NTNU and TIGER submitted iris methods.

Table 2. Results of the 6 methods submitted (error rate in %)
 Team
 Intermediate  Submission
no.
Rank GF2
GFRR@GFAR=0.01
EER
 TIGER 2 1 7.426.09
 NTNU3 12
9.61
6.25
 NTNU1 23
11.25
8.48
 NTNU2 14
17.62
11.38
 NTNU1_1 25
28.05
10.86
 NTNU4 26
93.28
37.70



1. An intermediate test set, with 20/40 users (for Periocular and Iris, respectively) was used to obtain the intermediate results.
2. All possible cross-sensor intra-class comparisons are implemented to evaluate the false non-match rate (FNMR).
3.Three samples were selected from each class for each sensor to evaluate the false match rate (FMR).

4. The metric GF2 given by GFRR@ GFAR=0.01(
same as GFRR@ GFAR=1%) will be used as the performance indicator to rank the submitted algorithms. (EER will be used in case of a tie.)
5. 
The procedure to deal with failures is compliant to the international standardization documents (namely, ISO/IEC 19795-1:2006).
Therefore, the evaluation was made using the error metrics GFAR and GFRR that consider both the failure-to-enroll (FTE) and the failure-to-acquire (FTA) rates.

Generalized false reject rate and generalized false accept rate:

GFAR = FMR * (1 – FTA) * (1 – FTE)

GFRR = FTE + (1 – FTE) * FTA + (1 – FTE) * (1 – FTA) * FNMR

(where FTA is the Failure to acquire rate and FTE is the failure to enrol rate, both according the ISO document).




Comments