causal
Test-Time Fine-Tuning Unlocks ARC Performance
Jack Cole's state-of-the-art LLM approach to ARC (35% with a 240M-parameter model) works only because of test-time fine-tuning—on-the-fly fine-tuning a version of the LLM for each test problem—which adds the active inference LLMs normally lack; without it, performance collapses to a negligible 1-2%.
causalpending
Speaker
François CholletEvidence Quote
“One thing that's really critical to making the model work at all is test time fine-tuning... What it's doing is trying to address one of the key limitations of LLMs today: the lack of active inference.”
Source
Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:26:16 AM
Your notes
Loading notes...