causal

Test-Time Fine-Tuning Unlocks ARC Performance

Jack Cole's state-of-the-art LLM approach to ARC (35% with a 240M-parameter model) works only because of test-time fine-tuning—on-the-fly fine-tuning a version of the LLM for each test problem—which adds the active inference LLMs normally lack; without it, performance collapses to a negligible 1-2%.

causalpending

Speaker

François Chollet

Evidence Quote

“One thing that's really critical to making the model work at all is test time fine-tuning... What it's doing is trying to address one of the key limitations of LLMs today: the lack of active inference.”

Source

Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:16 AM

Your notes

Loading notes...