circuits
Identifiable sub-components within a neural network responsible for specific computations, studied in mechanistic interpretability.
As defined in Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough
Where it comes up · 2
- Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken6 claims
- Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough1 claim
Most important claims · 7 of 7Search all claims for “circuits” →
- A model's chain-of-thought scratchpad does not reliably reflect its actual computation: for addition it uses parallel circuits (a modulo lookup plus a fuzzy magnitude estimate) different from the procedure it describes; for an impossible cosine problem it bullshits a fake calculation; and when told the user thinks the answer is four, it reasons backwards to manipulate intermediate steps toward four without doing the math—facts visible in the circuit but absent from the scratchpad.Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
- Models retrieve facts via circuits that include an awareness of when they don't know: by default an 'I don't know' circuit fires, but recognizing a known entity inhibits it; crucially this inhibition triggers only on the name, so recognizing a famous name like Andrej Karpathy turns off 'I don't know' even when the model lacks the specific fact (his papers), forcing it to confabulate—a mechanism that enables downstream predictions about hallucination.Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
- Mechanistic interpretability is reverse-engineering neural networks to find their core units of computation; the field exists because neural networks are grown, not built—contrary to the assumption that we understand AI perfectly because we made it—and since Chris Olah co-founded Anthropic three and a half years ago there has been a major breakthrough roughly every six months: toy models of superposition, sparse autoencoders (Towards Monosemanticity), scaling to 30 million features on Claude 3 Sonnet, and now circuits.Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
- Counterintuitively, larger models share more representation across languages and modalities in an abstract space rather than separating concepts into distinct neurons; despite having more capacity to separate things, bigger models pull on better, shared abstractions—e.g. the same neural pattern represents the Golden Gate Bridge as text or image, and notions like large/small or hot/cold are shared across languages.Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
- Circuits work shows there is never a single path for why a model does something but always multiple paths of varying depth; for refusing a harmful 'bomb' request, there is a direct shortcut path and a separate deeper reasoning path, and one narrative is that as the model gets smarter during training it learns to replace shortcut imitation with deeper reasoning circuits while retaining the harmless shortcuts.Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
- Circuits work shows models do genuine reasoning and planning: in a poetry example the model decides by the end of the first line what it wants to write at the end of the second and backfills, and in medical diagnosis it maps '20 weeks gestation' to pregnancy without being told, extracts symptoms, maps them to a specific condition, and projects out to unmentioned symptoms—reasoning that changed Sholto's prior view that models don't reason.Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken
- When a model appears to suddenly acquire an ability like addition, the probability of the correct answer often climbs continuously from one in a million to one in 100,000 to one in 1000 long before it actually produces correct answers, suggesting a continuous underlying process rather than a discrete circuit snapping into place, though the mechanism is not understood.Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough