causal

Superposition explains uninterpretable neurons

Models learn a compression strategy called superposition—packing more features than parameters into high-dimensional vectors—when data is high-dimensional and sparse, which is why individual neurons are polysemantic and confusing (firing for 'Chinese', 'fish', 'trees', and URL full stops), and projecting activations into a higher-dimensional space with a sparsity penalty undoes this compression to yield clean, monosemantic features.

causalpending

Speaker

Trenton Bricken

Evidence Quote

“the reason networks are so hard to interpret is in a large part because of this superposition... if you project the activations into a higher-dimensional space and provide a sparsity penalty, you get out very clean features”

Source

Sholto Douglas & Trenton Bricken — How LLMs actually think— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:31:11 AM

Your notes

Loading notes...