Hack and explore the latent space

Hack and explore the latent space

Not a complete challenge anymore, but under the umbrella of Hack the planets or Hack the stellar populations

Category: Unsupervised and self-supervised learning, visual analytics, clustering

Your challenge is to show nontrivial groupings of images in a (potentially multispectral) visual dataset of planetary surface images or astrophysical images by learning features in an unsupervised way. You can use a Python-based notebook we will provide to extract information from your dataset, or you could implement your own code/method. Mandatory: use the Latent Space Explorer tool for sake of understanding the task at hand and comparing results.

Sub-Tasks

1. Choose an image dataset: it could be formed by standard RGB images or multi-spectral images. (hint: use a multi-labeled dataset)

2. Train a model that extracts features (latent vectors), example:

1. Convolutional AutoEncoder (provided)

2. SimClr (provided)

3. One of your choice/implementation (not provided)

3. Load results of your experiments into NEANIAS NextCloud

4. Explore the representation space using Latent Space Explorer service

1. Reduce the dimensionality of the representation by using pca/tsne/umap (or implement on your PC an algorithm of your choice and visualize results)

2. Perform clustering algorithm on the dataset using one of the pre-provided algorithms or implement on your PC an algorithm of your choice

3. If present, visualize the ground-truth labels

5. Provide some metrics (confusion matrix, if ground truth is available) or qualitative observations about the results

Background

• NEANIAS Latent Space Explorer Service documentation

• AIxIA Discussion Paper – upcoming

• Google TensorFlow Blog: Introducing TensorFlow Similarity

• Latent Space Cartography

• Manifold learning

• Interpreting multispectral images

Indicative Data Sources

Usual suspects, increasing complexity and difficulty in evaluating quantitatively achieved results:

• MNIST dataset

• CelebA dataset (subsets of it)

• 17 Category Flower Dataset

• Multispectral Image Recognition (subsets of it)

Recommended Tools

• Latent Space Explorer