causal
Architecture Progress Frees Blocked Compute
Algorithmic progress like the transformer should be understood not as increasing the power of the blob of compute but as removing artificial hindrances that older architectures impose; the big blob of compute hypothesis has core factors (parameters, compute, data quantity, data quality, loss function) plus enabling conditions like proper symmetries (convolutions for translation, attention over long context) and good numerical conditioning — if compute is artificially blocked or closed off from the distant past, 'the spice doesn't flow' and it won't work.
causalpending
Speaker
Dario Amodei (Anthropic CEO)Evidence Quote
“the models want to learn, the compute wants to be free and it's being blocked in various ways where you don't understand that it's being blocked until you need to free it up.”
Source
Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:25:28 AM
Your notes
Loading notes...