Evaluation Metrics

For the performance evaluation, we selected the recently standardized ISO/IEC 30107-3 metrics: Attack Presentation Classification Error Rate (APCER), Normal/Bona Fide Presentation Classification Error Rate (NPCER/BPCER) and Average Classification Error Rate (ACER) as the evaluation metric, in which APCER and BPCER/NPCER are used to measure the error rate of fake or live samples, respectively. Inspired by face recognition, the Receiver Operating Characteristic (ROC) curve is introduced for large-scale face Anti-spoofing in our dataset, which can be used to select a suitable threshold to trade off the False Acceptance Rate (FAR) and False Rejection Rate (FRR) according to the requirement of real applications.

We will also give other metrics, such as 1-FRR (meaning correct acceptance rate) @FAR=10E-2, 10E-3, and ACER. About the definition of ROC, FAR, and FRR, please refer to [1].

Other metrics used for face Anti-spoofing will also be given:

(Positive samples mean attack ones, and negative samples mean Real/Bona Fide samples.)

• Attack Presentation Classification Error Rate (APCER ):

           APCER = FN / (TP + FN)

• Normal (Bona Fide) Presentation Classification Error Rate (NPCER/BPCER ): 

            NPCER/BPCER= FP/(FP + TN)

• Average Classification Error Rate (ACER):

          ACER = (APCER + NPCER) / 2

• False Rejection Rate (FRR):

          FRR =   FN/(TP+FN)

• False Acceptance Rate (FAR):

           FAR =  FP/(FP+TN)

Note:

TP: The attacks are recognized as the attacks; 

TN: The real samples are recognized as the real ones;

FP: The real samples are recognized as the attacks;

FN: The attacks are recognized as the real samples;

ref:

[1] Aghajan, H., Augusto, J. C., & Delgado, R. L. C. (Eds.). (2009). Human-centric interfaces for ambient intelligence. Academic Press. [link]