causal

Garden of Forking Paths Inflates False Positives

Because there are many different analyses a researcher could run on any dataset and only one needs to be statistically significant to get published, the true chance of finding a 'significant' result under the null is far higher than 5%—like a lottery where you keep buying tickets until you win—even without any conscious cheating.

causalpending

Speaker

Andrew Gelman

Evidence Quote

“there are many different analyses that you could do of any data set. And you kind of only need one that's statistically significant to get it to be publishable. It's a little bit like you have a lottery where there's a 1 in 20 chance of a winning ticket, but you get to keep buying lottery tickets until you win.”

Source

Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths— EconTalk
Created: 6/13/2026, 12:23:45 AM

Your notes

Loading notes...