A Closer Look at Few-shot Classification
ICLR 2019
Carnegie Mellon University
Georgia Tech
Georgia Tech
National Taiwan Univerisity
Virginia Tech
Figure: Meta-learning few-shot classification algorithms.
Abstract
Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples. While significant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difficult. In this paper, we present 1) a consistent comparative analysis of several representative few-shot classification algorithms, with results showing that deeper backbones significantly reduce the gap across methods when domain differences are limited, 2) a slightly modified baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the mini-ImageNet and the CUB datasets, and 3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classification algorithms. Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones. In a realistic, cross-domain evaluation setting, we show that a baseline method with a standard fine-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.
Baseline & Baseline++
Our baseline is to simply exploit a linear layer and a softmax layer.
Our baseline++ is to classify a image by measuring its cosine distance to each weight vector. Each weight vector can be interpreted as prototype of a class.
Training strategy of both baseline and baseine++ could be divided into training stage (on only base classes) and fine-tuning stage (on only novel classes).
For more details, please check our paper.
Figure: Baseline and Baseline++ few-shot classification methods.
Figure: Few-shot classification accuracy vs. network capacity (backbone depth).
Accuracy vs. Network Capacity
We observe that gaps among different methods diminish as the backbone gets deeper. We list detailed statistics in the paper.
Effect of Domain Difference
Here we demonstrate the result of 5-shot accuracy in self-domain/cross-domain scenarios with a ResNet-18 backbone. The Baseline models perform relative well with larger domain differences.
Figure: 5-shot accuracy in different scenarios with a ResNet-18.
Table: 5-way meta-training and N-way meta-testing experiment.
More-way in Meta-testing Stage
We further conduct 5-way meta-training and N-way meta-testing experiment on mini-ImageNet. Baseline++ compares favorably against other methods in both shallow or deeper backbone settings.
Paper
Code
Citation
@inproceedings{chen19closerfewshot, author = {Chen, Wei-Yu and Liu, Yen-Cheng and Kira, Zsolt and Wang, Yu-Chiang and Huang, Jia-Bin}, title = {A Closer Look at Few-shot Classification}, booktitle = {International Conference on Learning Representations}, year = {2019}}