dictionary learning
An unsupervised technique (also called sparse coding) that projects model activations into a higher-dimensional sparse space to recover interpretable features.
As defined in Sholto Douglas & Trenton Bricken — How LLMs actually think
Where it comes up · 2
Most important claims · 5 of 5Search all claims for “dictionary learning” →
- Models learn a compression strategy called superposition—packing more features than parameters into high-dimensional vectors—when data is high-dimensional and sparse, which is why individual neurons are polysemantic and confusing (firing for 'Chinese', 'fish', 'trees', and URL full stops), and projecting activations into a higher-dimensional space with a sparsity penalty undoes this compression to yield clean, monosemantic features.Sholto Douglas & Trenton Bricken — How LLMs actually think
- Feature splitting means a model will learn as many features as you give it capacity for—low capacity yields a coarse 'bird' feature, higher capacity splits into ravens, eagles, sparrows—so for scalable interpretability you can find the coarse direction (e.g. biology) and selectively expand only that region (depth-first search) rather than exhaustively projecting to a million dimensions (breadth-first search).Sholto Douglas & Trenton Bricken — How LLMs actually think
- Dictionary learning is done after training by feeding the model many inputs, taking activations, and projecting them into a higher-dimensional space and back, with a sparsity constraint—it is unsupervised in that you are not telling the model what features to find, only asking it to reconstruct sparsely, and you label the features post hoc, which is its advantage over linear probes that require knowing what you're looking for.Sholto Douglas & Trenton Bricken — How LLMs actually think
- Dictionary learning via sparse autoencoders extracts interpretable monosemantic features from polysemantic networks where none were apparent before — falling out cleanly without specifying categories in advance — which serves as non-trivial validation of the linear representation and superposition hypotheses and scales to production models like Claude 3 Sonnet.Dario Amodei
- Sparse autoencoders are like telescopes revealing features, but there is strong evidence we still see only a small fraction of them — a kind of 'dark matter' in the neural network universe that may never be observable or computationally tractable to observe, which has implications for safety if a significant fraction of networks remains inaccessible.Dario Amodei