Slides & videos:

Slides and videos of our tutorial are made available on the Program page.

About the Tutorial

Many vision applications require identifying objects and object-related information in images. Such identification can be performed at different levels of detail, which are addressed by different visual detection tasks such as “object detection” for identifying labels of objects and boxes bounding them, “keypoint detection” for finding keypoints on objects, “instance segmentation” for identifying the classes of objects and localizing them with masks, and “panoptic segmentation” for both semantic segmentation of background classes and instance segmentation of objects. Accurately evaluating performances of these methods is crucial for developing better solutions.

Accordingly, in this tutorial, we aim to extensively delve into the evaluation of visual detectors. Within the scope of our tutorial, we will first cover the basics of evaluating visual detectors in order to allow someone not familiar with visual detection to grasp the basics. Then, we will introduce the Localisation Recall Precision (LRP) Error [1,2] and present thorough comparative both theoretical and comparative analyses with Average Precision (AP) and Panoptic Quality (PQ) [3] on various visual detection tasks. Finally, we will discuss bridging the gap between training and evaluation by directly optimizing AP and LRP, which involves a non-differentiable ranking step that is difficult to optimize using conventional gradient-based methods.

References

[1] K. Oksuz, B. C. Cam, S. Kalkan*, E. Akbas*, "One Metric to Measure them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks", IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), in press, 2022. [Paper] [Code]

[2] K. Oksuz, B. C. Cam, E. Akbas, S. Kalkan, "Localization Recall Precision (LRP): A New Performance Metric for Object Detection", European Conference on Computer Vision (ECCV), pp. 521-537, Springer, 2018. [Paper] [Code]

[3] Kirillov, A., He, K., Girshick, R., Rother, C., & Dollár, P. "Panoptic segmentation". IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9404-9413), 2019.