factual

GPT-3 Scored Zero on ARC in 2020

In the original 2020 Kaggle competition for ARC (with a ~$20K prize), people tried GPT-3 on the public data shortly after its release and it scored zero, and the contest taught that there is no obvious shortcut to hack the benchmark.

factualpending

Speaker

François Chollet

Evidence Quote

“What the first contest taught us is that there is no obvious shortcut.”

Source

Francois Chollet — Why the biggest AI models can't solve simple puzzles— Dwarkesh Patel Podcast
Created: 6/13/2026, 3:26:16 AM

Your notes

Loading notes...