Few-shot Learning

Few-shot Learning is a machine learning paradigm where models are trained to make accurate predictions or decisions with very limited examples. It includes two special cases known as One-shot Learning and Zero-shot Learning.

One-shot Learning

One-shot Learning involves training models with only one example per class. The goal is to enable the model to generalize and recognize new instances of classes with minimal training data.

• Siamese Networks - Utilizes a twin neural network architecture to learn a similarity metric between pairs of input samples.

• Triplet Networks - Similar to Siamese networks but involves three instances (anchor, positive, and negative) in each training example. The model is trained to minimize the distance between the anchor and positive examples while maximizing the distance between the anchor and negative examples.

• Prototypical Networks - Learns a representation space where the mean of a few examples of each class (prototype) can be used for classification.

• Meta-Learning (or Learning to Learn) - Meta-learning algorithms train models to quickly adapt to new tasks with minimal data. Examples include Model-Agnostic Meta-Learning (MAML) and Reptile.

• Matching Networks - Utilizes attention mechanisms to enable the model to weigh the importance of different examples when making predictions.

• Nearest-Neighbor Methods - Simple yet effective, nearest-neighbor methods classify new instances based on their similarity to a few examples from each class.

• Hallucination-Based Approaches - Augment the training data by generating additional examples for classes with limited samples.

Zero-shot Learning

Zero-shot Learning goes a step further, aiming to train models to recognize classes for which they have seen no examples during training. This involves learning semantic relationships and attributes that generalize across classes.

• Semantic Embeddings - Utilizes semantic embeddings to represent classes in a continuous vector space. The model is trained to predict the relationship between visual features and semantic attributes.

  • Example: In natural language processing, word embeddings like Word2Vec or GloVe capture semantic relationships between words. Zero-shot learning can be achieved by associating these embeddings with visual or other types of data, enabling the model to generalize to unseen classes.

• Attribute-Based Methods - Associates attributes with classes and learns to predict the presence or absence of these attributes in images.

  • Example: Identifying birds by training a model to detect attributes of birds such as a beak or feathers.

• Graph-Based Methods - Constructs a semantic graph where nodes represent classes and edges encode semantic relationships. The model leverages this graph structure to make predictions for unseen classes.

  • Example: In a zero-shot setting, the model predicts the class of an animal by considering its position and connections in the semantic graph (e.g., hierarchical relationships between different animal species).