Does Mechanistic Interpretability Scale?
Evidence from Multiple Choice Capabilities in Chinchilla
Figure 13: Projection of query and key deltas of L40 H62 on the first three principal components
Figure 13: Projection of query and key deltas of L40 H62 on the first three principal components
(This is an interactive plot. You can click and drag to rotate, and scroll to zoom. You can also click on legend entries to show/hide parts of the data.)