Pre-training does not require thoughtful selection of training data because it uses all available data, whereas RL training requires explicit choices about which environments to create, introducing many degrees of freedom and opportunities for misalignment with actual capabilities.

factualpending

Speaker

Ilya Sutskever

Evidence Quote

“When you do pre-training, the question of what data to train on was answered, because that answer was everything... But when people do RL training, they do need to think.”

Source

Ilya Sutskever – We're moving from the age of scaling to the age of research— Dwarkesh Patel
Created: 8/12/2026, 6:07:19 PM

Your notes

Loading notes...