All visualisations below, apart from the last one, are not associated with any of the experiments and are purely aimed at communicating how to read the axes alignment results.
Let's assume that we have a 3-dimensional latent space in which we have projected all data points with their corresponding color label. We assume that the axis alignment criterion is satisfied and the color factor of variation is aligned with Z0. In order to evaluate how good the alignment is, we perform Principal Component Analysis (PCA) on the latent cluster for each label and examine the alignment between the resultant eigen vectors/principal components (Cs) and the basis vectors (Zs) that span the latent space.
It is worth noting that from all the possible Z/C pairs, we don't need to inspect all. If we take the all blue points, for example, they would seem like a spherical blob which has compressed along Z0. Thus, the smallest eigen vector - C0 - would be aligned with (parallel to) Z0. The rest of the eigen vectors, orthogonal to C0, would span the non-compressed part of the sphere, which has equal variations in all directions, and can therefore be any tuple of mutually-orthogonal vectors, not necessarily aligned in any way with Z1 and Z2 - see side view above. The same rationale can be extended to any N-dimensional space, where we would have an N-dimensional sphere compressed along one of the dimensions and not along the rest.
Correspondence between the vector pairs and the reported cosine similarity diagrams. The Black & White scheme should guide the reader's attention - white cells are important and mark the cosine similarities between the smallest eigen vector with all basis vectors and between its most-parallel basis vector and all other principal components. In each diagram the size of square (i,j) - represents the cosine distance between Zi and Cj.
Axes Alignment evaluation for the full model (a), the classifier baseline (b) and the β-VAE (c) for the 3 experiments - (1), (2) and (3) respectively. For each experiment and each model, the cosine similarity diagram for each label is shown. E denotes the average entropy estimates over normalised white cells values for a single model and a single experiment. For each concept group in each experiment-model combination we should observe the same white cell patterns if axis alignment is being achieved.
We work with a 4-dimensional latent space: |Z| = 4.