causal

Autoregression Struggles With Unlikely Connections

Autoregressive next-token prediction is a strange way to generate mathematics, because the substantive cross-field connections are by nature unlikely next tokens, so training on predictable continuation may suppress them.

causalpending

Speaker

Grant Sanderson

Evidence Quote

“The process of repeatedly predicting something is just pretty different from how you would think as a writer to compose it and think it through.”

Source

Dwarkesh Patel Podcast - Grant Sanderson – AI and the future of math— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:11:54 AM

Your notes

Loading notes...