We train the DML models with 3 approaches (ProxyNCA++, ProxyAnchor, SoftTriple) by ourself, each trained 5 trails with 5 seeds.

Surprisingly, we find that although the losses proposed by 3 approaches are distinct, they share similar generalization errors.

The top 10 wrong classes reported by different approaches have high overlap with each other.

In the following figure, each row represents a training trail with a specific seed. The first 5 rows are 5 runs from ProxyNCA++, next 5 rows are ProxyAnchor, last 5 rows are from SoftTriple. Each column represents a testing class. If the testing class is inside the top 10 most frequently wrong testing classes, it is highlighted in colours of red/blue/green. We observe the highlighted columns are well aligned between different runs, different approaches, which indicates they are sharing similar generalization errors.