Pre-training scaling is slowing down because 9-10 orders of magnitude of progress since AlexNet means most low-hanging fruit has been picked, requiring research breakthroughs like reasoning, synthetic data, tool use, and AI scaffolding to continue intelligence scaling.
causalpending
Speaker
SoniaEvidence Quote
“pre-training does seem to be slowing down. We've scaled pre-training by nine or 10 orders of magnitude since the Alexet days, and that means a lot of the lowhanging fruit has been picked...The research ecosystem is finding new ways to break through. The most important breakthrough was reasoning from OpenAI.”
Created: 8/11/2026, 7:13:14 AM
Your notes
Loading notes...