factual
GPT-3 Scored Zero on ARC in 2020
In the original 2020 Kaggle competition for ARC (with a ~$20K prize), people tried GPT-3 on the public data shortly after its release and it scored zero, and the contest taught that there is no obvious shortcut to hack the benchmark.
factualpending
Speaker
François CholletEvidence Quote
“What the first contest taught us is that there is no obvious shortcut.”
Source
Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel PodcastCreated: 6/13/2026, 3:26:16 AM
Your notes
Loading notes...