factual

LLMs Exhibit Non-Classical Compositional Representation

Emerging mechanistic interpretability evidence suggests LLMs implement a non-classical form of compositional representation: words are vectors in high-dimensional space, and attention modules read and write information into distinct subspaces—analogous to content-addressable memory—tracking syntactic and semantic relationships separately without the discrete concatenation of classical symbolic structure.

factualpending

Speaker

Raphael Milliere

Evidence Quote

“it's doing a more sophisticated form of composition that can still keep certain attributes of the different words neatly separated by reading and writing information to distinct subspaces.”

Source

230 | Raphaël Millière on How Artificial Intelligence Thinks— Sean Carroll's Mindscape
Created: 6/13/2026, 12:19:51 AM

Your notes

Loading notes...