factual
LLMs Exhibit Non-Classical Compositional Representation
Emerging mechanistic interpretability evidence suggests LLMs implement a non-classical form of compositional representation: words are vectors in high-dimensional space, and attention modules read and write information into distinct subspaces—analogous to content-addressable memory—tracking syntactic and semantic relationships separately without the discrete concatenation of classical symbolic structure.
factualpending
Speaker
Raphael MilliereEvidence Quote
“it's doing a more sophisticated form of composition that can still keep certain attributes of the different words neatly separated by reading and writing information to distinct subspaces.”
Created: 6/13/2026, 12:19:51 AM
Your notes
Loading notes...