RL environments
Task setups designed by teams for reinforcement learning training, each defining problems and reward signals.
As defined in Ilya Sutskever – We're moving from the age of scaling to the age of research
Other definitions
Simulated, isolated task settings where a model acts like a worker and receives a measurable reward for progress on a verifiable problem.
Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]
Where it comes up · 4
- Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job4 claims
- An audio version of my blog post, Thoughts on AI progress (Dec 2025)1 claim
- Ilya Sutskever – We're moving from the age of scaling to the age of research1 claim
- Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]1 claim
Most important claims · 7 of 7Search all claims for “RL environments” →
- Either models will soon learn on the job in a self-directed way, which makes the entire mid-training supply chain of pre-baking skills (like navigating browsers or using Excel) pointless, or they won't, which means AGI is not imminent; humans never go through a special phase rehearsing every piece of software they might use.An audio version of my blog post, Thoughts on AI progress (Dec 2025)
- Because RL training requires choosing which environments to train on, researchers inadvertently take inspiration from the evals they want to look good on, effectively making the human researchers the real reward hackers; combined with inadequate generalization this explains the gap between eval and real-world performance.Ilya Sutskever – We're moving from the age of scaling to the age of research
- The workaround of building clones of Slack, Gmail, and other common applications for computer-use training is currently very labor-intensive and unscalable, though it could scale once AIs can code high-fidelity clones themselves.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job
- Having AIs rebuild complex applications from scratch is a strong RL objective for coding, and it doubles as a way to build the application clones needed for computer-use training.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job
- The current research bet of the labs is that training AIs on millions of verifiable tasks across thousands of diverse RL environments will basically produce AGI, because such training builds general problem-solving skills like making sustained progress on open-ended tasks despite errors and ambiguity.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job
- The current paradigm is like a genius grad student who never gets an internship: labs keep supplying classroom case studies through RL environments rather than letting the AI learn from the accumulated experience of broad deployment.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job
- Frontier labs used to spend most on data but now spend far more on RL environments, which capture recursive self-improvement on verifiable tasks and are the recipe for improvement on any task with a self-grading system.Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]