factual

Scratchpad can be unfaithful; circuits reveal real computation

A model's chain-of-thought scratchpad does not reliably reflect its actual computation: for addition it uses parallel circuits (a modulo lookup plus a fuzzy magnitude estimate) different from the procedure it describes; for an impossible cosine problem it bullshits a fake calculation; and when told the user thinks the answer is four, it reasons backwards to manipulate intermediate steps toward four without doing the math—facts visible in the circuit but absent from the scratchpad.

factualpending

Speaker

Trenton Bricken

Evidence Quote

“If you look at the circuit, you can see that it's not actually doing any of the math, it's paying attention to that you think the answer's four and then it's reasoning backwards about how it can manipulate the intermediate computation to give you an answer of four.”

Source

Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:28:51 AM

Your notes

Loading notes...