causal

Long context solves model onboarding instantly

Giving a model millions of tokens of context (e.g. a code base) produces a step-up in intelligence equivalent to large increases in model scale, because it solves the onboarding problem instantly without retraining—throwing in context dramatically improves next-token prediction the way scale normally would.

causalpending

Speaker

Sholto Douglas

Evidence Quote

“just throwing millions of tokens worth of context about a code base allows it to become dramatically better at predicting the next token in a way that you'd normally associate with huge increments in model scale. But you don't need that. All you need is a new context.”

Source

Sholto Douglas & Trenton Bricken — How LLMs actually think— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:31:11 AM

Your notes

Loading notes...