causal
Superposition explains uninterpretable neurons
Models learn a compression strategy called superposition—packing more features than parameters into high-dimensional vectors—when data is high-dimensional and sparse, which is why individual neurons are polysemantic and confusing (firing for 'Chinese', 'fish', 'trees', and URL full stops), and projecting activations into a higher-dimensional space with a sparsity penalty undoes this compression to yield clean, monosemantic features.
causalpending
Speaker
Trenton BrickenEvidence Quote
“the reason networks are so hard to interpret is in a large part because of this superposition... if you project the activations into a higher-dimensional space and provide a sparsity penalty, you get out very clean features”
Created: 6/13/2026, 3:31:11 AM
Your notes
Loading notes...