causal

Embers of Autoregression Bias

Raw language models like GPT-4 are very sensitive to the probability of the outputs they must produce: they are better at counting strings of 30 letters than 29 because the number 30 appears more often on the internet than 29, so when nearby high-probability answers exist the model produces the high-probability one rather than the correct one—an idiosyncratic bias the authors called 'embers of autoregression.'

causalpending

Speaker

Tom Griffiths

Evidence Quote

“it sort of produces the high probability thing rather than the thing that it's supposed to produce. And so that's a sort of like weird idiosyncratic bias of language models.”

Source

343 | Tom Griffiths on The Laws of Thought— Sean Carroll's Mindscape
Created: 6/13/2026, 12:12:58 AM

Your notes

Loading notes...