causal

Benchmarks and Training Environments Are the Same Thing

Anything that can be turned into a benchmark can also be a training environment, so tasks like conjecture generation that resist benchmarking also resist current RLVR training, which requires a fundamentally different approach.

causalpending

Speaker

Dwarkesh Patel

Evidence Quote

“The kinds of things you can’t make benchmarks for are also the kinds of things, at least in the current paradigm, you can’t easily train for. There’s really no fundamental difference between a benchmark and a training environment.”

Source

Dwarkesh Patel Podcast - Grant Sanderson – AI and the future of math— Dwarkesh Patel Podcast
Created: 11/17/2026, 6:11:54 AM

Your notes

Loading notes...