forecast
RL generalizes like the GPT-2-to-GPT-4 fine-tune transition
Current RL jaggedness mirrors the GPT-2 era when fine-tuned models excelled only at their fine-tuned tasks; just as scaling pre-training compute led GPT-4 to generalize across sub-tasks better than smaller fine-tuned models, scaling total RL compute will produce the same transition to unsupervised meta-learning and broad generalization across reasoning domains—the interp agent generalizing without ever being RL'd for interp is early evidence.
forecastpending
Speaker
Sholto DouglasEvidence Quote
“I think right now what we're seeing with RL is pretty much the same story playing out. There's this jaggedness of things that they're particularly trained at. But as we expand the total amount of compute that we do RL with, you'll start to see the same transition”
Created: 6/13/2026, 3:28:51 AM
Your notes
Loading notes...