causal

Replications Fail Because Original Studies Were Uncontrolled

When replications fail, the key difference is usually not the experimental conditions but that the replication is controlled—the analysis is specified in advance—whereas the original study was uncontrolled and permitted p-hacking; even honest researchers like Nosek discovered they had p-hacked without realizing it.

causalpending

Speaker

Andrew Gelman

Evidence Quote

“typically the main way the new and the old studies differ is that the new study is controlled... you can do p-hacking on the old study but not on the new study.”

Source

Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths— EconTalk
Created: 6/13/2026, 12:23:45 AM

Your notes

Loading notes...