causal

RL sucks supervision through a straw

Reinforcement learning is terrible because after a long rollout you get a single final reward bit, and every token along a successful trajectory gets upweighted equally—even the wrong alleys—broadcasting one noisy supervision signal across the whole trajectory; humans instead do far fewer attempts plus a rich review of which parts went well, which current LLMs have no equivalent of.

causalpending

Speaker

Andrej Karpathy

Evidence Quote

“you're sucking supervision through a straw”

Source

Andrej Karpathy — AGI is still a decade away— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:44 AM

Your notes

Loading notes...