causal
Long context solves model onboarding instantly
Giving a model millions of tokens of context (e.g. a code base) produces a step-up in intelligence equivalent to large increases in model scale, because it solves the onboarding problem instantly without retraining—throwing in context dramatically improves next-token prediction the way scale normally would.
causalpending
Speaker
Sholto DouglasEvidence Quote
“just throwing millions of tokens worth of context about a code base allows it to become dramatically better at predicting the next token in a way that you'd normally associate with huge increments in model scale. But you don't need that. All you need is a new context.”
Created: 6/13/2026, 3:31:11 AM
Your notes
Loading notes...