causal

Prediction-accuracy culture drives benchmark hacking

Because the current era treats prediction accuracy as everything, researchers engage in benchmark hacking (also called state-of-the-art hacking): tuning systems to top leaderboards by tiny, often not statistically significant margins, prioritizing engineering for the toolbox over understanding the phenomenon, its data sources, and its uncertainties.

causalpending

Speaker

Tina Eliassi-Rad

Evidence Quote

“it's basically benchmark hacking or state of the art hacking. And that's basically what is going on, that's the reality of it.”

Source

301 | Tina Eliassi-Rad on Al, Networks, and Epistemic Instability— Sean Carroll's Mindscape
Created: 6/13/2026, 12:26:13 AM

Your notes

Loading notes...