grindable
A property of training domains where many parallel rollouts can be run against a deterministic and replayable simulator from identical starting points, allowing for efficient data collection
As defined in 20p5-kQXF_Q
Where it comes up · 1
Most important claims · 4 of 4Search all claims for “grindable” →
- A domain must be not only verifiable but also grindable, meaning you can run many parallel rollouts against a deterministic and replayable simulator; verifiability alone does not make a domain amenable to RL progress.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job
- Computer-use agents cannot be trained by having a thousand agents attempt the same checkout flow on a live site like Amazon, because the site detects and shuts down the bots.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job
- The workaround of building clones of Slack, Gmail, and other common applications for computer-use training is currently very labor-intensive and unscalable, though it could scale once AIs can code high-fidelity clones themselves.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job
- Coding is easy to train with RL because one can create an environment with a software repo missing a feature and let a thousand parallel agents each work on an identical copy of the container.Dwarkesh Patel Podcast - The next big breakthrough will be AIs learning on the job