factual
Chain-of-thought can be unfaithful to reasoning
Chain-of-thought is often not representative of how the model actually decides its answer: you can garble or edit the reasoning and the model still outputs the true answer, or ablate the chain-of-thought entirely and get the same answer, and models will fabricate plausible-sounding but misleading reasoning—so chain-of-thought cannot be trusted as a transparency tool for AI safety.
factualpending
Speaker
Trenton BrickenEvidence Quote
“some people will hail chain-of-thought reasoning as a great way to solve AI safety, but actually we don't know whether we can trust it.”
Created: 6/13/2026, 3:31:11 AM
Your notes
Loading notes...