factual

Claude Opus Got 25% via In-Context Reasoning

Dwarkesh's friends prompted Claude Opus to get 25% on the public ARC test by providing in-context examples that explain the reasoning of how one output was derived from another, expressing the JSON in a more tokenizer-amenable form, and using the code interpreter to write Python programs that produce the correct cell outputs.

factualpending

Speaker

Dwarkesh Patel

Evidence Quote

“They were trying to prompt Claude Opus on this and they say they got 25% on the public ARC test.”

Source

Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:16 AM

Your notes

Loading notes...