causal
Pre-training to easy reward signals enables long-horizon RL
Counterintuitively, longer-horizon tasks can be easier to judge if you can ever get the reward—'make money on the internet' is an easy reward signal to evaluate—so the key is being able to pre-train up to easy-to-judge end signals (does the website work, do people like it); if you could pre-train on every screen in America, the achievable RL tasks differ radically from those possible using only today's internet.
causalpending
Speaker
Sholto DouglasEvidence Quote
“if you could pre-train up to the easy to judge reward signals, does your website work? Does it go down, do people like it?”
Created: 6/13/2026, 3:28:51 AM
Your notes
Loading notes...