factual

Feature splitting enables depth-first interpretability

Feature splitting means a model will learn as many features as you give it capacity for—low capacity yields a coarse 'bird' feature, higher capacity splits into ravens, eagles, sparrows—so for scalable interpretability you can find the coarse direction (e.g. biology) and selectively expand only that region (depth-first search) rather than exhaustively projecting to a million dimensions (breadth-first search).

factualpending

Speaker

Trenton Bricken

Evidence Quote

“you can find the direction that the biology feature is pointing in, which again is very coarse, and then selectively search around that space. So only do dictionary learning, if something in the direction of the biology feature fires first.”

Source

Sholto Douglas & Trenton Bricken — How LLMs actually think— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:31:11 AM

Your notes

Loading notes...