definition

Imitation Learning Is Short-Horizon RL

Imitation learning is not categorically different from RL; it is just short-horizon RL where the episode is a single token long: the LLM conjectures the next token from its world understanding and receives reward proportional to how well it predicted the true token.

definitionpending

Speaker

Dwarkesh Patel

Evidence Quote

“Imitation learning is just short horizon RL. The episode is a token long.”

Source

Some thoughts on the Sutton interview— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:01 AM

Your notes

Loading notes...