Superposition Hypothesis

Definition

The idea that models, being underparameterized relative to the high-dimensional sparse structure of their data, learn to pack more features than they have neurons by representing them as overlapping directions in activation space, producing polysemantic neurons.

Cast within

No topic-region cast yet — this appears once Superposition Hypothesis's compiled claims are aligned into a topic region's argument tree.

Claims mentioning Superposition Hypothesis (3)

Your notes

Loading notes...