causal

P-value hacking causes replication crisis

If you try many model specifications and report only the one that confirms your hypothesis with a standard p-value, the reported significance is wrong because the p-value ignores the many specifications searched; this 'p-value hacking' has produced a replication crisis, with large fractions of studies in some psychology subfields failing to replicate.

causalpending

Speaker

Susan Athey

Evidence Quote

“if you report it with a p-value that says, 'Oh, this is highly unlikely to have happened,' but you actually tried a hundred to get this, that's an incorrect p-value. And we call that p-value hacking.”

Source

Susan Athey on Machine Learning, Big Data, and Causation— EconTalk
Created: 6/13/2026, 3:35:37 AM

Your notes

Loading notes...