definition

Dictionary learning is unsupervised feature discovery

Dictionary learning is done after training by feeding the model many inputs, taking activations, and projecting them into a higher-dimensional space and back, with a sparsity constraint—it is unsupervised in that you are not telling the model what features to find, only asking it to reconstruct sparsely, and you label the features post hoc, which is its advantage over linear probes that require knowing what you're looking for.

definitionpending

Speaker

Trenton Bricken

Evidence Quote

“You need to label them post hoc, but it's unsupervised. You're just like, “give me the features that explain your behavior.””

Source

Sholto Douglas & Trenton Bricken — How LLMs actually think— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:31:11 AM

Your notes

Loading notes...