factual
Automated labs would design better small-scale science
Automated researchers could develop a much better science of learning from small experiments, extending scaling laws and discovering phenomena like muP, which predicts optimal hyperparameters for larger runs from models 100x smaller, and GPT-4's loss was predicted from models trained with 10,000x less compute.
factualpending
Speaker
Dwarkesh PatelEvidence Quote
“the GPT-4 paper says that they were able to predict that model’s final loss using models trained with 10,000x less compute.”
Source
Dwarkesh Patel Podcast - The avalanche that hit math is coming for AI development— Dwarkesh Patel PodcastCreated: 11/17/2026, 6:27:26 AM
Your notes
Loading notes...