FINAL RESULTS


Five teams submitted their methods to the competition: IDIAP, IITIndore, HH, NTNU and TIGER.

Results of the PERIOCULAR TASK
IDIAP, IITIndore, HH, NTNU and TIGER submitted periocular methods.

Table 1. Results of the 12 methods submitted (error rate in %)
 MethodRank
 GF2
GFRR@GFAR=0.01
 EER Enrolment Processing Time (sec)
 Matching Processing Time (sec)
 HH110.74
0.82
17.95
0.45
NTNU1 21.86
1.59
0.84
1.23
 IDIAP23
2.03
1.65
7.18
2.91
 IDIAP34
2.03
1.55
6.95
2.14
 NTNU254.19
2.75
0.81
1.19
 NTNU36
5.94
3.21
1.09
1.00
 IITIndore1* 76.09
4.53
9.89
6.19
 IDIAP18
6.24
3.46
2.80
2.49
 IITIndore2* 9
6.48
5.20
 8.416.54
  HH2 1014.87
9.16
17.18
0.06
 HH3 1116.28
10.65
15.63
0.04
 TIGER 1245.44
12.25
10.73
0.02
* This submission was made in a posterior date to the deadline agreed by the participants and the organizers.

Results of the IRIS TASK
NTNU and TIGER submitted iris methods.

Table 2. Results of the 5 methods submitted (error rate in %)
 TeamRank GF2
GFRR@GFAR=0.01
EER
 Enrolment Processing Time
(sec)
 Matching Processing Time
(sec)
 NTNU4 10.00
0.05
 57.98 0.06
NTNU3
 28.43
5.58
 1.011.08
NTNU13
8.81
6.19
 0.871.31
TIGER
4
10.637.84   1.69 0.07 
 NTNU25
12.77
7.95
59.09
0.05



1. The final test set, with 55/110 users (for Periocular and Iris, respectively) was used to obtain the final results.
2. All possible cross-sensor intra-class comparisons are implemented to evaluate the false non-match rate (FNMR).
3.Three samples were selected from each class for each sensor to evaluate the false match rate (FMR).

4. The metric GF2 given by GFRR@ GFAR=0.01(
same as GFRR@ GFAR=1%) will be used as the performance indicator to rank the submitted algorithms. (EER will be used in case of a tie.)
5. 
The procedure to deal with failures is compliant to the international standardization documents (namely, ISO/IEC 19795-1:2006).
Therefore, the evaluation was made using the error metrics GFAR and GFRR that consider both the failure-to-enroll (FTE) and the failure-to-acquire (FTA) rates.

Generalized false reject rate and generalized false accept rate:

GFAR = FMR * (1 – FTA) * (1 – FTE)

GFRR = FTE + (1 – FTE) * FTA + (1 – FTE) * (1 – FTA) * FNMR

(where FTA is the Failure to acquire rate and FTE is the failure to enrol rate, both according the ISO document).




Comments