factual

Caesar Cipher Reveals Memorized Not General Algorithm

LLMs can solve a Caesar cipher for common shift values like three or five (which appear frequently in internet examples) but fail for an arbitrary value like nine, proving they have memorized specific cases rather than the generalized algorithm—if they could synthesize the solver on the fly, the shift value would not matter since it doesn't increase problem complexity.

factualpending

Speaker

François Chollet

Evidence Quote

“It does not encode the generalized form of the algorithm, but only specific cases... If it could actually synthesize on the fly the solver algorithm, then the value of n would not matter at all, because it does not increase the problem complexity.”

Source

Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:16 AM

Your notes

Loading notes...