causal
RL sucks supervision through a straw
Reinforcement learning is terrible because after a long rollout you get a single final reward bit, and every token along a successful trajectory gets upweighted equally—even the wrong alleys—broadcasting one noisy supervision signal across the whole trajectory; humans instead do far fewer attempts plus a rich review of which parts went well, which current LLMs have no equivalent of.
causalpending
Speaker
Andrej KarpathyEvidence Quote
“you're sucking supervision through a straw”
Created: 6/13/2026, 3:26:44 AM
Your notes
Loading notes...