PaMM: Pose-aware Multi-shot Matching for Improving Person Re-identification
Yeong-Jun Cho¹² and Kuk-Jin Yoon²
¹Gwangju Institute of Science and Technology (GIST),
²Korea Advanced Institute of Science and Technology (KAIST)
IEEE Transections on Image Processing (TIP) 2018
IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR) 2016
Notice
The codes are very old version (written in MATLAB). No maintenance for them. Sorry for this...! 😿
ABSTRACT
Person re-identification is the problem of recognizing people across different images or videos with non-overlapping views. Although a significant progress has been made in person re-identification over the last decade, it remains a challenging task because the appearances of people can seem extremely different across diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person re-identification by analyzing camera viewpoints and person poses called pose-aware multi-shot matching. It robustly estimates individual poses and efficiently performs multi-shot matching based on the pose information. The experimental results obtained by using public person re-identification data sets show that the proposed methods outperform the current state-of-the-art methods, and are promising for accomplishing person re-identification under diverse viewpoints and pose variances.
OVERVIEW
Person re-identification is quite challenging due to camera viewpoint and person pose variations. However, what if the camera viewpoint and the pose priors of people in every non-overlapping camera are known in advance? In fact, progress in auto-calibration techniques has enabled the extraction of additional cues such as camera parameters, ground plane, and the 3D position of people without any offline calibration tasks. Exploiting those additional cues permits the estimation of the poses of people. This study fully exploits those additional cues for multi-shot matching and proposes the Pose-aware Multi-shot Matching (PaMM) for efficient person re-identification.
Suppose that upon estimating camera viewpoints and the poses of people, there is a simple 2 vs. 2 matching scenario that contains one same-pose matching (front–front) and three different-pose matchings (front–right, left–front, left–right). The result of the same-pose matching can generally be expected to be more reliable than those of different-pose matchings, because people maintain their appearance between cameras when their poses are the same (This work excludes photometric issues such as illumination changes and camera color response differences). Then, in such a multi-shot matching scenario, it is desirable that same-pose matching (front–front) plays a more important role than different-pose matchings. Hence, this work incorporates this matching idea by aggregating the matching scores of all pose matchings in a weighted summation manner, where the thicknesses of the lines indicate the matching weights. We also study how to efficiently train matching weights and match between multi-shot appearances using pose information.
Figure 1. The proposed pose-aware multi-shot matching (PaMM) framework for person re-identification. First, the pose of each person is estimated. Second, a multi-pose model is generated. Finally two multi-pose models are matched based on pre-trained matching weights. The thicknesses of lines indicate the matching weights.
RESULTS
We extensively evaluated and compared the performance of the proposed method using public person re-identification datasets such as 3DPeS, PRID 2011, iLIDS-VID and MARS.
Table 1. Performance comparision results with dataset 3DPeS.†Denotes a multi-shot mathing method.The best and the second best scores in each rank are marked with BOLD and BLUE. AUC is an area under a curve of CMC
Figure 2. Qualitative analysis results of PaMM. The true matches are highlighted with red boxes. 3DPES, PRID 2011, iLIDS-VID
🖇️ Citation
@inproceedings{cho2016improving,
title={Improving person re-identification via pose-aware multi-shot matching},
author={Cho, Yeong-Jun and Yoon, Kuk-Jin},
booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition}, pages={1354--1362},
year={2016} }
@article{cho2018pamm,
title={PaMM: Pose-aware multi-shot matching for improving person re-identification},
author={Cho, Yeong-Jun and Yoon, Kuk-Jin},
journal={IEEE Transactions on Image Processing}, volume={27}, number={8}, pages={3739--3752},
year={2018}, publisher={IEEE} }
