causal

Significant Estimates in Small Samples Are Biased Upward

In a small, highly variable sample the standard error is large, so any result that can reach statistical significance must be large (about two standard errors from zero); because only large estimates clear the significance bar, published significant effects are systematically overestimated due to selection bias.

causalpending

Speaker

Andrew Gelman

Evidence Quote

“when you report things that are statistically significant, by definition, or by construction, a statistically significant estimate has to be large. Under typical calculations, it has to be at least 2 times, 2 standard errors away from zero.”

Source

Andrew Gelman on Social Science, Small Samples, and the Garden of the Forking Paths— EconTalk
Created: 6/13/2026, 12:23:45 AM

Your notes

Loading notes...