causal

Next-word training supports planning ahead

Next-word prediction as a training objective and generation method is compatible with planning: the Anthropic rhyme example shows a model generating the final word 'rabbit' before earlier words, revealed by mechanistic interpretability, so autoregressive training can encapsulate staged problem-solving.

causalpending

Speaker

Chandra Sripada

Evidence Quote

“that next-word generation as a generation method, as well as a closely aligned training objective, is quite compatible with the model generating things that come much later early on”

Source

363 | Chandra Sripada on How LLMs and Humans are Cognitive Cousins— Sean Carroll's Mindscape
Created: 11/17/2026, 6:53:48 AM

Your notes

Loading notes...